The most vexing, complicated and elusive question in international relations is how to achieve an order, based on rules, that enjoys legitimacy, rewards investments in cooperation, reconciles clashing interests and deters conflict. It is not a problem over which a magic wand can be waved. But in our own time, immense and patient efforts have been made towards that general goal, however imperfect the result.

The concept of the ‘rules-based international order’ refers today in its most general sense to arrangements put into place to allow for cooperative efforts in addressing geopolitical, economic and other global challenges, and to arbitrate disputes. It is embodied in a variety of multilateral institutions, starting with the United Nations and running through various functional architectures such as the Bretton Woods system, the corpus of international law and other regimes and treaties, down to various regional instances where sovereignty is pooled or where powers have been delegated consensually by states on a particular issue.

Some aspects of the rules-based order are heavily informed by distinct values, such as those contained in the Universal Declaration of Human Rights. But, more often than not, they simply prescribe a set of basic principles for how the business of international political and economic relations is to be transacted. The parameters of legitimate and illegitimate behaviour are specified. Compliance is incentivized, and some scope to sanction transgressors is provided for.

For some, the rules-based international order is a politically highly charged concept. Indeed, the absence of a common standardized definition of it is perhaps a by-product of the controversy which the mere notion of a rules-based order often attracts – among those who had no or little part in its shaping; those who regard multilateralism as an infringement of sovereignty and a straitjacket on national ambitions; and those who sense in it a presumption of universal values and shared interests that jars with their own particular historical experience and political preferences. And in a world in which each country occupies its own place on the spectrum of attraction to, tolerance of and resistance to multilateralism, it is inevitable that the present system should be a patchy and incomplete one.

If that patchiness seems increasingly apparent today, then this reflects the proliferation of problems on a truly global scale that multilateral initiatives have as yet failed to keep up with. This is partly because of the sheer pace of change and the deep complexity of problems, and partly because any significant programme of coordinated action requires a focus and consensus that today is in shrinking supply.

More than that, some of the sharpest challenges – climate change; the lack or weakness of rules in the sea, space and cyber domains; the dilemmas thrown up by technological change – are problematic precisely because they are areas in and through which geopolitical competitions are being contested. The policy challenges may be new, but the pattern of behaviour currently surrounding them presents some dangerous echoes from the past.

Throughout history, most attempts to form international orders have been conceived in a coercive way. From classical antiquity to the 20th century, the dominant form of order has been that imposed or attempted by successive territorial empires, or by predominant powers who made the rules by fiat and were deferred to by their neighbours and satellites.

Significant attempts at more collaborative conceptions of order, aimed at coexistence and minimizing risk through rules and accepted conventions, have been far rarer. And the key point about them is that they have been attempted only after competition has spilled over in an uncontrolled, exhausting and ruinous conflict that has called for mechanisms and understandings to prevent a recurrence of disaster. That, in any case, has been the European experience, and subsequently the result of the engulfing crises that radiated out globally from Europe in the 20th century.

Early efforts at order-building focused on mutual recognition and the management of what were felt to be inevitable rivalries. The Westphalian Peace of 1648 emerged from a 30-year period of religious war in Europe. It emphasized the sanctity of sovereignty and non-interference in the internal affairs of other states as a precondition for order, but relied on a jostling balance-of-power approach to the preservation of a basic stability.

A tolerance of conflicts to correct imbalances was implicit to the scheme. But its acute sensitivity to shifts in alignments of power contributed to the later conflicts – from the wars of the Spanish Succession and Austrian Succession to the Seven Years’ War – that ravaged Europe in the 18th century and occurred in an increasingly global theatre of military operations, tracing the development of European imperial projects.

Despite these shortcomings, the balance-of-power model was produced again as a remedy to uncontrolled conflict, at the Congress of Vienna in 1814–15, following more than 20 years of French Revolutionary and Napoleonic wars. A Concert of Europe, accommodating a rehabilitated France, was instituted to regulate the system and periodically decide major geopolitical issues. But it fell into disuse. And although Europe did not suffer a general war for the rest of the 19th century, the salient geopolitical facts were ones not of power balances but of the sharp relative decline of France and the vertiginous rise of Prussia, which defeated Austria and France on the path to German unification.

These dynamics produced convoluted and ever-widening balancing manoeuvres that by the eve of the First World War in 1914 had congealed and hardened into the opposing Triple Alliance and Triple Entente systems, which trapped their respective members into tangled commitments to fight at the trigger of a crisis.

The peacemaking efforts, in Paris in 1919, that followed the war entailed conscious efforts to overturn the balance-of-power model. The tone was set by US President Woodrow Wilson’s Fourteen Points, with their emphasis on transparency and openness, while the concepts of egalitarianism among states, the drive towards disarmament and the practice of collective security were central to the revolutionary creation of a League of Nations in 1920.

But the peacemaking also included a punitive dimension – the designation of German culpability, the demand of economic reparations and territorial adjustments – imposed by victor on vanquished. To its critics, the international order being evolved, and the rules drafted to underpin it, had the attributes of an involuntary settlement more than those of a construct built by equals.

Lacking a comprehensive membership – crucially, the US had demurred, while other major powers progressively withdrew or were thrown out – and the military means to impose itself, a divided and often circumspect League faltered in meeting a succession of international crises. It then collided fatally with the revanchism of Germany, Italy and Japan that produced the Second World War.

The ambitiousness and eventual institutional intricacy of the UN system founded in 1945 marked a response to the scale of the ordeal through which the world had passed, and sought to correct the deficits of the League. The UN’s membership and the activity of its main organs and specialized agencies all grew prodigiously in succeeding decades, as did its efforts to advance the spirit and culture of multilateralism.

But by giving special privileges to the victors, principally through veto rights held among a small group of permanent Security Council members, the UN reflected and perpetuated a certain historical circumstance: there was no formal institutional adaptation in its highest structures to account for a progressive redistribution of international power, the rehabilitation of defeated countries, the rise of the decolonized world or the desire of emerging powers to assume international responsibilities commensurate with their heft. Rather than a mechanism for international governance, it remained an intergovernmental body through which states pursued their specific or collective priorities.

Indeed, the dominant questions around order in the first five decades of the UN’s existence were those posed by the Cold War conducted by the US and the Soviet Union and their respective allies and satellites, while the UN in effect was a prominent arena in which this global antagonism was carried out.

The world order was bipolar in concentrating power in two camps, with a swath of neutrals, non-aligned and swing players in between; and bi-systemic in the complete contrast in the ideological affinities and economic models that were promoted. Nuclear weapons raised the stakes associated with direct conflict to an existential level, and so pushed armed contests to peripheral theatres or on to skirmishing proxies.

The collapse of communism in the early 1990s ushered in a new dispensation. Those who divined the arrival of a ‘unipolar moment’ for the US were perhaps more accurate in their choice of epithet than they knew. At least on the surface, the US became by far the preponderant power. The decline and 1991 dissolution of the Soviet Union, in consequence of its economic decrepitude and strategic overstretch, not only removed the US’s peer competitor, but also opened up avenues for promoting economic liberalization and democratic government.

This shift was manifest in particular in changing dynamics in Europe. The US had sponsored the reunification of Germany and was a patron of its subsequent embedding in an integrating, democratic and liberal region. Over time, this drew the former Warsaw Pact members into EU and NATO structures (albeit at a pace and with a completeness that Russia’s strategic calculations could not be accommodated to).

And yet, despite these advances, in retrospect the chief development of the 20 years after the Cold War was a different one: globalization had at a gathering pace prompted a redistribution of political power, while its interlocking economic structures created a dense web of interests and dependencies that moved in all directions. It was likely in these circumstances that the appearance of any major emergency would produce insistent voices demanding what they saw as a more inclusive, legitimate and effective form of international order.

Crises duly arrived, first in the shape of the 2003 US-led invasion of Iraq, which strained alliances and stirred controversial debates about the justice and permissibility of military interventions and the need for constraints on US power; and then in the form of the financial meltdown of 2008, seen by many as a principally Western debacle calling for new global economic governance structures as instanced in the improvised G20. Neither set of debates was conclusively resolved, but each persisted against the backdrop of quickening systemic change.

The dilemmas about the shape and maintenance of a rules-based order with multilateralism at its core have since only deepened. The world is pulling in different directions. The ‘America First’ posture of the Trump administration has upturned the central feature of the system. It entails a distaste for multilateral agreements, a disavowal of traditional notions of US leadership, and an insistence on the unimpeded exercise of American power in pursuit of defined national interests.

China asserts the centrality of multilateralism, and practises it selectively, but on the whole favours binary diplomatic transactions where it holds asymmetric advantages; it has used this approach in the construction of its Belt and Road Initiative, as well as on other fronts.

Europe has created in its continent a rules-based order par excellence in the shape of the EU, but its energy has been sapped and its introversion fed by a succession of crises, of which the amputation of the Brexit-bound UK is simply one. The EU has yet to chart its future course or define a global strategy to uphold and advance the multilateralism which has been at its core.

Russia unabashedly is subverting the rules-based order as part of a programme of aggrieved self-aggrandizement. Japan champions the principle of a rules-based system, but the country has been disoriented by its abrupt detachment on this issue from its traditional US partner; while Japan has sought to engage like-minded countries in the West, they have not forged a concerted practical plan of action together.

Among other regional powers, Brazil has a populist government that echoes many of the Trump administration’s instincts, and India, whatever its preferences, has yet to acquire a foreign policy or presence on the global stage equal to its demographic weight and economic potential.

Prominent points of risk in this fragmenting picture are the multilateral trade system, efforts to address climate change, and collective measures to deal with entrenched conflicts.

One obvious consequence of the attrition of the rules-based system through the indifference or ambitions of the great powers is that it will leave smaller states much more exposed and hostage to the vagaries of geopolitical competition. A key question therefore is whether such states will choose and be able to defend a system which gives them a measure of protection.

Over recent decades, a variety of regional groupings – ASEAN, the African Union, the Gulf Cooperation Council, the Organization of American States – have evolved as species of rules-based mechanisms and in order to gather their collective weight. They make a ready constituency for those who would build a coalition for multilateralism. But it is also clear that the support of smaller regional players for such an approach depends on a revision of the rule-making system towards greater inclusivity and a broader say as to the issues it should address.

It is in the context of these trends and structural shifts that Chatham House Expert Perspectives 2019 offers ideas for how to modernize and adapt elements of the rules-based international order. As the title of this opening essay indicates, the imperative to ‘adapt’ reflects the gravity of contemporary challenges, and the inability of many existing structures to underpin ever-more-essential cooperation. Chatham House experts do not offer a master plan, but they attack the problem from a variety of indicative angles.

Suggestions are offered as to where gaps in international rules – regarding economic governance, the global health architecture and in respect of under-regulated domains such as space, for example – need to be filled to address immediate problems and advertise the relevance of multilateralism.

Other ideas demonstrate how logjams affecting some aspects of the system can be worked around; how key powers with scope to shape the system should be engaged; how a broader variety of actors beyond national governments need to be drawn into the effort; how rule-breakers might be tackled; and how imposing order on some chaotic situations requires the fundamental premises of existing policies to be rethought.

Chatham House, which celebrates its centenary in 2020, is a child of efforts after the Great War to reconceive the conduct of international relations and fulfil a mission that is today defined as the creation of a ‘sustainably secure, prosperous and just world’. The historical record shows that international orders not built on these attributes will fail.

This essay was produced for the 2019 edition of Chatham House Expert Perspectives – our annual survey of risks and opportunities in global affairs – in which our researchers identify areas where the current sets of rules, institutions and mechanisms for peaceful international cooperation are falling short, and present ideas for reform and modernization.