Cultures are defensive constructions against chaos, designed to reduce the impact of randomness on experience. They are adaptive responses, just as feathers are for birds and fur is for mammals.

Mihaly Csikszentmihalyi. Flow: The Psychology of Optimal Experience (1990) at 81.

Introduction

Progress in the last 200 years

We are living right now (in March 2024) through what feels like the most turbulent social and political moment in my lifetime, with major wars being fought in both Europe and Asia, resulting food and energy insecurity, economic recessions in Germany, Japan, and China, high inflation, and at the same time major technological revolutions in aerospace and defense, biotechnology, and artificial intelligence. For those born after the 1960s who have wondered what it might have felt like to have lived through that era, the post-covid era has provided answers.

In the midst of chaos, we are programmed to seek order. Whether this programming is somehow inherent in the laws of nature is an interesting question to be left for another day. The question I explore here is what explains the exceptional progress we have seen over the past 200 years. It seems worth asking that question now, as we are in the middle of unraveling so much of that progress, both in the United States and in the rest of the world.

But before we get to the unraveling, have a closer look at what I mean by progress over the past 200 years. Hans Rosling, a doctor and public health advocate, provided a vivid and detailed numerical visualization in this 4 minute video:

The Gapminder Foundation website also provides an interactive and updated version of the chart shown in the video, which allows you to explore the data yourself. Of particular interest for what follows are the trajectories of the United States, the United Kingdom, France, and the handful of other countries that accelerated up and to the right fastest during the 20th century. Trajectories of each country over the entire 200 year span can be traced by clicking the “Trails” button on the right.

Besides life expectancy and GDP per capita, numerous other measurements of progress might have been made. In each case, we could at least in principle drill down into the details, to analyze the multifactor causes of observed increases in each measurement from each country by referring to specific political, economic, and technological developments. But the purpose of this essay is to ask instead about the broad trend across all of these countries over the past 200 years. The question to be answered is this:

What caused the acceleration in progress we observed beginning 200 years ago?

The Orthodox Answers

There are answers to this question that correspond to three distinct sets of what might be called orthodox beliefs. These three orthodoxies can be labeled with a single word as “technology,” “democracy,” and “capitalism.” Thousands of books and articles have been written in support of each of these orthodoxies. Here I provide a short description of each to provide a working definition. In each description, I rely upon a contrast with opposing belief systems to limit the definition.

Technology

Probably easiest defined as opposition to luddism or asceticism. Best articulated recently by Marc Andreessen recently in his Techno-Optimist Manifesto, this is the belief that progress is driven at least mostly by improvements in technology, which acts as an accelerant to providing larger quantities and higher quality goods and services to people. The belief in technology correlates well with the timing of acceleration in progress over the last 200 years, with the last 75 years especially seeing a lot of cutting edge technological development clustered in the United States and the San Francisco Bay Area, in particular.

Democracy

Probably easiest defined as opposition to monarchy, dictatorship, and totalitarianism. Best articulated in Why Nations Fail: The Origins of Power, Prosperity, and Poverty (2012) by Daron Acemoglu and James A. Robinson, this is the belief that inclusive political and economic institutions that enforce property rights, create a level playing field, and encourage investments in new technologies and skills are the root cause of economic success. The belief in democracy and democratic institutions correlates well with economic progress over the last 1,000 years, but doesn't as obviously explain the acceleration of progress over the last 200 years.

Capitalism

Probably easiest defined in opposition to socialism or communism. Best articulated by Adam Smith in An Inquiry into the Nature and Causes of the Wealth of Nations, this is the belief that the self-interested actions of individuals in a free market, as if guided by an "invisible hand," leads to the most efficient allocation of resources and drives economic progress for society as a whole. This strikes me as the most compelling of the orthodox beliefs because it correlates well both temporally and geographically with progress over the last 200 years. For example, the economic progress in Singapore since the 1960s is difficult to explain in terms of technological development or democratic institutions, but nobody would question whether Singapore has been hospitable to capitalists during that period. Conversely, geographical areas where communism was dominant have trailed economic progress in the rest of the world consistently, with only a few possible exceptions (such as China under Deng Xiaoping or Vietnam more recently), which nonetheless can be explained in terms of the embrace of capitalist institutions under a non-democratic regime.

Gaps in the Orthodox Answers

As compelling as they may be when considering specific cases in isolation, the orthodox answers leave important gaps in explaining what caused the acceleration in progress we observed beginning 200 years ago.

For example, Germany was dominant in developing cutting edge technology in the mid-19th century, but fell behind as National Socialism turned its people against themselves and their neighbors, driving key talent (especially Jewish) away (especially to the United States). Iceland ranks among the most democratic countries on the planet, but is not widely recognized as having exceeded any other country in its pace of economic development. As noted above, the embrace of capitalism seems to have more explanatory power, but in considering the case of specific countries it becomes clear that establishment of capitalist institutions (including for-profit corporations and large-scale competitive markets for goods and services) is itself contingent upon the development of certain government institutions, including the freedom to incorporate and engage in impersonal transactions with guaranteed minimum rights (such as the ability to enforce contracts or bring claims against a counterparty for fraud in an impartial forum).

The complete explanation for what has driven progress in the last 200 years isn't to be found in any single one of these orthodox beliefs, but in some combination of them.

Technology as Question Begging

At the very least, technology should be viewed as both a cause and consequence of progress rather than by itself a sufficient cause. The reason for this is simple: On many occasions throughout recorded history before the last 200 years, important technological advances were made, which were not by themselves sufficient to drive the kind of progress we have seen in the last 200 years. Although China was well ahead of medieval Europe in technological development, authoritarian restrictions on trade in the 15th century substantially interrupted economic progress, eventually leaving China behind Europe in technology. Similarly, xenophobic policies in post-Renaissance Europe hampered economic progress there. Xenophobia by elites and violent factionalism among elites set these regions back decades or centuries behind where they might otherwise have ended up.

Democracy as Benevolent Dictatorship

A country led by an elected executive who serves as a broker of the interests of an elite oligarchy might be considered a democracy, but whether that democracy promotes progress depends on the vision of the executive and her ability to harness the interests in the direction of progress rather than on the fact that she was elected. Lee Kwan Yew was successful in Singapore. Putin’s predecessors were not in Russia.

Contingent Capitalism

Among the orthodoxies, capitalism does provide the most compelling explanation of what caused the acceleration in progress we observed beginning 200 years ago. As a group, the handful of countries in which we saw the most rapid progress in the last 200 years have been those countries that have embraced open competition and markets relatively free from government intervention. But if we define capitalism only in opposition to socialism or communism — that is, as free competition among for-profit companies without government intervention — then even the orthodoxy of capitalism is incomplete as an explanation of what caused the progress we observed in the past 200 years.

While it is true that progress was more rapid in countries that put fewer constraints on for-profit companies, there is no country in which for-profit companies competed in the absence of any government constraints. In fact, the for-profit companies operating in any given country as corporations are formally recognized by a sovereign authority, and constrained by a complex web of legal regulations that provide, among other things, a shield to shareholders from personal liability beyond their capital invested and tax efficiencies to pooling of capital. The problem with orthodox beliefs in capitalism is that orthodox believers tend to ignore the constitutive role that governments have played in bringing about economic competition.

But rather than argue about whether or to what extent a belief in capitalism should be defined in opposition to government — a debate that has been going on for more than 50 years among, for example, Austrian and Chicago School economists — I turn now to the more interesting question of what blend of government and markets seems to work best in promoting progress.

Chaotic Progress through Open Competition

This question is central in Violence and Social Orders by Douglass C. North, John Joseph Wallis, and Barry R. Weingast. (I’ll call them “NWW” in the remainder of this essay.)

At the most abstract level, their thesis can be understood as motivated by the observation that legal systems serve as a substrate for impersonal trust. Providing some constraints around the rights of counterparties to an impersonal transaction helps increase the number and frequency of those transactions. Capitalism is therefore contingent upon the development of an equally robust competition among political parties for control of the legal system that has the power to determine the rules for economic competition. Progress under a system of capitalism will be chaotic. There is no easy or quick path to establishing such a system; even after such a system has been established, its survival remains contingent upon a dynamic balance between free competition among political parties and corporations. The ensuing state of progress, if indeed it obtains at all, might be called “chaotic progress.”

NWW distinguish between two "social orders" that have been used by groups of people to control violence since the development of agriculture in the neolithic period:

• Natural States, in which violence is controlled through a coalition of personal relationships among elites who control access to resources, including violence itself. In natural states, the ability of non-elites to form organizations is limited. Violence is limited because and to the extent it is bad for the elite coaltion.
• Open Access States, in which violence is controlled through "powerful, consolidated military and police organizations subservient to the political system," which is open to entry and may be contested by any group. The competition among groups for control of the political system is subject to a set of rules that limit illegitimate violence so that only the groups that can get and maintain broad support stay in control of the political system. Violence is limited by deterrence (through threat of punishment by the state) and by depriving non-state organizations of access to violence.

An organization of organizations (namely, "government") maintains a monopoly on violence in both natural states and open access states, but the the organization of organizations is a coalition controlled by wealthy elites in natural states; in open access states, by contrast, the organization of organizations is a constantly evolving coalition of interest groups formed and dissolved to aggregate political support from people who might have no personal relationships with each other.

Crucially, organizations in open access states are impersonal: the identity of the organization is independent of the identity of its individual members. Free entry, or the ability for anybody to form organizations to exploit economic and political opportunities, results in vigorous competition that drives economic and political benefits out to larger and more dispersed groups of people within society. Except for violence — which remains a government monopoly — these organizations are free to compete with each other. Elites still work hard to control resources and benefit themselves in open access states, but because any group of people is allowed to form a coalition, a coalition of elites must compete for access to the political system and its monopoly on violence, and cannot win without concessions that would in natural states be unnecessary.

Open Entry to Political and Economic Competition

How does open access obtain? NWW observe that in addition to meeting what they call “doorstep conditions” — namely, a perpetually lived state and consolidated political control over violence — an open access state has the following characteristics:

1. A widely held set of beliefs about the inclusion of and equality for all citizens.
2. Entry into economic, political, religious, and educational activities without restraint.
3. Support for organizational forms in each activity that is open to all (for example, contract enforcement).
4. Rule of law enforced impartially for all citizens.
5. Impersonal exchange.

NWW at 114. If you consider yourself a capitalist and libertarian, take a look at this list and ask yourself: “What can be removed?” If you answered 1, then you would have failed to recognize the constitution of the United States. If you answered 2, then can you really call yourself a capitalist or libertarian? If you answered 3, 4, or 5, then how would you support impersonal transactions at scale?

These five characteristics may look like an overfit to the dominant models for capitalism we have right now, but there is nothing that can be left out. And if you accept that, then you must also recognize that chaotic progress is not entirely a function of free economic competition, but also of free political competition to support 1, 2, 3, and 4. The chaotic progress we’ve seen over the past 200 years has been contingent upon a certain kind of government.

The Stability of Open Access States

Suppose we accept that a move from natural to open access states would be good for progress. How are two vigorous competitions among political parties and corporations supposed to keep a state stable? For NWW, the answer is a shift to the “double balance” of, on the one hand, power shared among the network of elites and, on the other hand, power shared between the network of elites and the non-elites. See NWW at 111-112.
In any open access society, open access in all systems is mutually reinforcing through three mechanisms:

1. “Everybody counts or nobody counts” (With thanks to Detective Bosch.) Citizens share beliefs in sharing, equality, and universal inclusion.

• All open access states have institutions and policies that share gains from markets and reduce risks of market participation. Examples include universal education, social insurance, and widespread infrastructure.
• Because these institutionalized beliefs result in sharing of market gains, they reduce demands for redistributions that can be crippling to markets.

2. Schumpeterian democracy Political parties compete for control through elections.

• The success of party competition in policing elites in power depends on a dense set of organizations representing a wide range of interests that can mobilize widely dispersed constituencies to punish attempts to limit access or extract rents from particular groups.
• These characteristics of democracy in the United States were also well articulated by Richard Posner in Law, Pragmatism & Democracy

3. Negative Feedback Institutions and incentive systems impose costs on incumbents who seek to limit access or extract rents.

• A shrinking economy and tax revenue results in a shift of power to opposing political parties.
• Mobile resources leave the country, weakening the state’s competitive position in international markets.

Because of 1, governments in open access states are bigger than governments in natural states. But because of 3, these governments tend to be more complementary to than controlling of markets.

As NWW argue at 133-135, these mechanisms result in stability through continuous dynamic experimentation and their ability to generate and maintain credible commitments between elites and non-elites.

Continuous experimentation results as the free flow of ideas within open access states generates a range of potential ways to understand and resolve new problems. The larger the problems faced by a society, the more extensive is both the debate about the nature of the problems and the set of potential solutions. Individuals and organizations affected by the problems have incentives to invest in creating and advertising new solutions. Political parties, interest groups, and organizations all compete to solve major problems and address crises. Those in power seek solutions to help them remain in power. Opposition parties and their support groups have strong incentives to expose the weaknesses in the incumbents’ proposals and to devise more attractive alternatives. The larger the problem, the more individuals and organizations are affected by it, and the more widespread public input and discussion. The free and open expression of ideas means that many ideas will be heard. In their quest to maintain or to regain power, competing parties will draw on this competition for solutions, seeking ideas that further their interests. Progress is power in an open access state.

The ability to generate and maintain credible commitments between elites and non-elites is also crucial to the stability of open access states. Political conflicts arise in all societies, for example, between rich and poor, agriculture and industry, workers and firms, and among different regions. The ability to make credible commitments in open access orders combines with this order’s ability to deliver impersonal benefits to widen the set of feasible solutions to conflicts. Open access orders therefore more readily address these conflicts without disorder. For example, students of Western Europe argue that explicit incorporation of labor allows credible commitments to policies that solve budgetary problems in times of fiscal stress. In contrast, a study of Argentina shows that this inability plagues natural states’ ability to solve similar problems.

Corporations as Legal Conspiracies

The productivity gains from organizing people into groups can be superlinear. As Richard Posner recognized in The Problems of Jurisprudence, this is implicitly why conspiracies are punished severely under criminal law. A corporation is in some sense a conspiracy that has been deemed legal for some narrow purpose, but early on they were often treated by proponents of capitalism with the same skepticism shown to the criminal kind.

For example, Adam Smith himself was skeptical of corporations. “Although much of the debate about Smith's view of corporations focused on his view about their efficiency, Smith saw corporations in a traditional Whig manner as grants of economic privilege used to secure political advantage. [His low opinion of corporations in general reflected less on the economic and organizational aspects of joint-stock businesses than on the natural state’s political effects of chartering -- the corrosive effects of corporate privileges given to towns, guilds, and monopolies.] As late as 1776, the founder of modern economics viewed corporations largely in natural-state terms -- as tools for the political manipulation of the economy.” NWW at 205.

But almost immediately after Adam Smith published The Wealth of Nations, a key enabler of chaotic progress evolved: After the 1780s, “a new institution developed, called general incorporation, which allowed the creation of a corporation through an administrative procedure without the explicit approval of a political body. General incorporation developed first in the United states. A statute that creates such a procedure is a general incorporation act.” NWW at 213.

The ability for anybody to form a political party came even later, and even in the United States as the Constitution was drafted, the Founders were deeply skeptical of the benefits of competition among political parties. But by the end of the Civil War, the vigorous competition among political parties had proved itself as a mechanism at least capable of constraining some otherwise violent competition among powerful interests.

“Individual citizens wanted banks, canals, and railroads to raise the value of their lands, help get their crops to market, and more closely integrate them into the American economy and society. They feared that those corporations would undermine and ultimately destroy the democratic political process they also valued. The solution hit upon by the Americans was not to eliminate corporations but to eliminate privileges by opening entry into the corporate form to anyone who wanted to form a corporation.” NWW at 238. Thus, “[a]lthough no decisive year or decade exists of when growth began to accelerate, nonetheless the onset of modern economic growth rates, per capita income growth in the neighborhood of one to 1.5% per year, does not occur in any of the countries until after 1840. After the 1850s, and with modest ups and downs, this steady growth occurred until the present day, with the exceptions of wars and the depressions of the 1920s and 1930s.” NWW at 247

NWW also provide an in-depth analysis of how open access states evolved in England and France as well as the United States. Well worth the time if you’ve read this far.

Summary and Conclusion

Neither technology, nor democracy, nor capitalism alone has been sufficient to drive the progress we have seen in the last 200 years. Rather, as NWW argue in Violence and Social Orders, that progress has been due to the special combination of democratic ideals and capitalism that has emerged in open access states through their credible commitments, which have maintained dynamic stability through non-violent resolution of political conflict and vigorous economic competition. In contrast, natural states face challenges in making credible commitments to potential new entrants, ultimately driving both economic and political instability and violence. The transformation of corporations from royal privileges into drivers of chaotic progress through the introduction of general incorporation acts and the transformation of political competition through political parties are an important part of the story.

Open access states exhibit dynamic stability because competition and experimentation lead to diverse solutions for problems, while credible commitments resolve conflicts peacefully. Natural states, on the other hand, often lack the ability to make credible commitments, making conflict resolution more challenging.

The chaotic progress of the past 200 years has been contingent upon the emergence and maintenance of open access states. A stable open access state requires a delicate balance of power among elites and non-elites, supported by vigorous competition among political parties and corporations. With apologies to Benjamin Franklin, what sort of state has evolution given us? An open access state, if we can keep it.