Self-Regulation and Human Progress
How Society Gains When We Govern Less
Evan Osborne


Chapter 1

Problems and Responses

I maintain that Liberty is the chief cause of excellence.

—EDWARD BAINES JR. (1800–1890)1


Consider the following situations, and how best to address them:

  1. A major media figure claims that eating a certain kind of food may cause the (literal) consumer to incur a fatal disease. Sellers of the food, maybe believing the charge false and certainly worried about damage to their sales, wish to prevent the claim from gaining acceptance and to prevent future similar claims from being made.
  2. A property developer wishes to build single-family homes on undeveloped land, plus roads to service them. If he does so, people have more housing choices, but traffic, and crowding more generally, increase there.
  3. People with certain types of autoimmune problems become convinced that ingesting parasitic worms, ordinarily a dangerous thing to do, will reset their immune systems, restoring their health. The government, worried about the unproven efficacy and unknown danger of such treatment, then prohibits it.
  4. A scientist publishes a paper the truth of whose data come into question. The scientist asserts they are accurate.
  5. The central square of a European city laid out before the invention of the internal combustion engine now has many automobile accidents.

All of these situations are based on actual events, and in each, there is expected harm to be traded against potential gain no matter what we do. The first is based on a case involving Oprah Winfrey, who allowed people on her television show to make claims about the danger of consuming beef. These claims generated a lawsuit (which Ms. Winfrey ultimately won) by Texas cattle ranchers worried about “defamation” of their product. If people wrongly believe a false claim about the danger of consuming beef, it does indeed damage the livelihood of those ranchers, but to punish the person who made the claim will deter other people from publicly raising what they see as other serious problems for fear of incurring similar, misplaced (if the claim is true) punishment.

The trade-offs of urbanization and exurbanization have generated a movement known as “smart growth” (the phrase is telling) that proposes to contain “unplanned” development by creating more zoning rules to channel it in desirable directions. The third story, about the worms, is told in a book by Moisés Velasquez-Manoff on new thinking about why autoimmune diseases happen.2 The Americans he describes resort to visits to Mexico or subterranean markets in the United States to obtain treatment that is prohibited by U.S. authorities. As for the fourth example, disputed scientific papers, whose value depends on their authenticity, are of course far from rare. And finally, it is always possible to try to improve traffic safety by more tightly controlling driving behavior, but the more copious traffic regulations are, the more difficult it is to get where you wish to go. Despite predictions that without traffic regulations drivers will be heedless of other drivers, several European cities in recent years have ended all traffic rules concerning their central squares, and there is evidence that accident rates have subsequently gone down.3

In thinking about the best choice in situations such as these, people will have different views about what ought to be done. So how to decide? In advocating for a nonpolitical way of addressing situations such as these, where everyone is partly right and partly wrong, there are two factors I ask readers to accept as true:

  1. We live in a world of limits, and information is one of the most important of these limits.
  2. The human condition has improved over time, and dramatically so in the past several centuries.

The first factor means that people make mistakes, and the second that despite this, some of these mistakes have been dealt with. I suggest that making those mistakes is part of learning how to take care of social problems. It is common when thinking about the generation of such responses to speak of regulated versus unregulated processes. More precisely, it is common to describe a lack of political regulation as equivalent to an unregulated process. When the comparison is made in these terms, generally the conclusion drawn is unflattering to the latter. But this is a mistake. All human interactions are regulated in some way. The contest over how to more often elicit better choices is not between regulated and unregulated processes, but among different kinds of regulation.

Each of the cases I set out has at least two means of regulating the problem, and the contrast between those two is the subject of this book. In one case, something intrinsic to the process allows it to satisfactorily, even optimally (given the limits of what we can know), self-regulate. In the other, people either do not even conceive of or doubt the efficacy of whatever self-regulating processes exist. Thus, regulation has to be accomplished externally. Of course, from the broadest possible perspective, there is only one human system, and any regulatory processes, including political ones, are thus internal to that system. And there are many systems that can be characterized as externally regulated, yet not by the political system. Examples include hierarchical religions, where leadership reserves the right to interfere in the self-governance of local congregations (external regulation), versus denominations where each congregation has almost complete freedom of action, subject only to maintaining a sufficiently large number of members to persist (self-regulation). And there are also systems that contain a subsidiary self-regulating process and an external entity that can regulate it—for example, the family as the self-regulating entity, but in fact one that receives external rewards and punishment from political or religious authorities. Such relations that do not implicate the state are also interesting, but I do not discuss them here. Rather, this is a book about the intersection of politics with self-regulating processes.

The question of interest then involves human systems that might, but need not (and often do not), rely extensively for regulation on application of state power (which is how I use politics throughout). To modern readers, accustomed to believing that political regulation of a system, especially an economic system, can improve its performance, the idea of a self-regulating system is increasingly implausible as a society becomes more complex. There is, however, substantial historical reason to believe, as I seek to demonstrate, that as society becomes more complex, the inadequacies of political regulation, and therefore the need for self-regulation, actually grow. But if instead it is political regulation that grows, existing problems fail to be addressed effectively, generating more anger and in turn more political regulation. Opportunities for human advancement thus shrink.

We shall see in Chapter 2 that for much of human history, such regulation by the state, which often thoroughly controlled the economic, political, and social system through nothing more than rulers’ possession of sufficient military force to deter challengers for power, was accepted as the natural order of things. (The role of order is important throughout the book.) But even in that world, self-regulating social processes existed. The clearest example is language. The English that the average, say, Canadian or Briton speaks now is somewhat different from the English of the late eighteenth century, quite different from the English of Shakespeare’s time, and worlds apart from the English of the era of Beowulf. But no central authority has designed the evolution of English. People invent new words to describe phenomena that cannot be efficiently described by the existing vocabulary, and words that lose their informative value fall out of use. And yet lacking central direction, grammar and vocabulary nonetheless evolve to meet the needs of contemporary users.4 In short, language regulates itself. That the economy is important while language is not is no objection. Language is essential to human communication; it is difficult to imagine there being much in the way of commerce, science, or ideas without it.

What we eat largely self-regulates too. Although governments regulate food safety and require the publication of certain nutritional information, it would strike most people as preposterous if the government were not just to forbid the consumption of some foods but require the consumption of others. Indeed, even a limited version of such a system, like that of the mandatory communal dining in much of China during the first decades of the communist era, seems to readers used to autonomy in such matters to demonstrate the remarkable reach of communist totalitarianism: the government told Chinese what to eat and where and when to eat it.5 Instead, our diets have become ever more diverse as wealth and improved global transportation allow us not just to sample, and fall in love with, cuisines from distant places but to combine and recombine ingredients and styles from different places to create a constant flow of food innovations. An example is the “Chinese Mexican” cuisine that is found along the California-Mexico border; depending on whether it appeals to large numbers of people in other places, it may soon spread and be subject to further mutation.6 Cuisine, like language, even while essential to human survival, manages to do just fine without much in the way of political regulation.

The significant amount of socioeconomic political regulation today is widely accepted in many countries. But the arguments for socioeconomic self-regulation were radical in their day. This book tells that story in the context of the development of the broader idea of self-regulation, of which socioeconomic self-regulation is only a part. These ideas were generated between the seventeenth and nineteenth centuries in Europe. The argument for unregulated markets came about after very similar arguments were made for self-regulation in both science and communication. Socioeconomic activity, and to an extent science and communication, are now more subject to political regulation, despite the argument for trusting self-regulation being the same then as now, and in each case. The book argues for greater confidence in self-regulating social systems, particularly in the socioeconomic context.


Any argument for the superiority of self-regulating processes must ultimately appeal to utilitarian considerations. While one can certainly appeal to a right to be left alone by the state, a major impetus for the partial repeal of the liberal consensus of the late nineteenth century was based on, first, pointing out deficiencies in the operation of an “unregulated” economy, and, second, an immediate resort to politics as the proper way to remedy these deficiencies. It was, and is, too seldom asked whether the consequences of politics as they are actually practiced are likely to be better or worse than those that will result when participants in the system attempt to work out problems within that system. It is common enough, not least among economists, to assert that the market in particular “fails” in this or that realm in deciding how resources will be used and the resulting production “distributed” among the citizenry. But given the political system as we find it rather than as we would like it to be, it is precisely the feedback embedded within it that promotes the desired reactions from its participants, a property lacked by political regulation. In making this argument, I use a number of terms that I now set out.


What properties should a properly self-regulating social system possess? At a minimum, a system is a set of components with a specified, if perhaps subject to alteration, set of links among them. We wish to describe both the components and the links, and upon doing so may be able to make some statements about the system as a whole. In the solar system, for example, each planet and moon has characteristics such as mass, nature of the atmosphere, and so on. Once Newtonian laws of motion were sufficiently well understood, the properties of the links and thus the system as a whole could be stated: the planets revolve around the sun and the moons around their nearest planets with certain frequencies, for example. The biological system also has components in the forms of its many species. In varying combinations, they consume sunlight, atmospheric gases, and nutrients found in other species. It is the links generated among these various species, cooperative in some cases and predatory in others, that allow us to speak of a “system.” Within the Darwinian system writ large, many biologists now also like to speak of discrete subsystems, food webs or ecosystems, that can plausibly be partitioned from the rest of the larger system for analysis. We may thus speak of changes in one ecosystem brought about by, for example, the entry (perhaps human-enabled) of “invasive” species from other ecosystems.

Biologists think of species (and, more recently, genes) as the fundamental components of the Darwinian system. Depending on the purpose one has in analyzing the system elements, their atoms or subatomic particles can be usefully thought of similarly for the natural world. But in the sort of comparison among social systems envisioned here, humans must be the fundamental unit because they are the ones who interact with other humans in every social system. Analogous to the distinction drawn between an ecosystem and the global biological system, this is not to say that the system must encompass the entire human population. We may speak of the “French socioeconomy” or the system of artistic expression we call music. (Throughout I use the term socioeconomy where appropriate in lieu of economy to describe the full spectrum of potentially self-regulating human activities, not all of them involving monetary commerce.) But even in such subsystems, humans are what species and atoms often are in natural systems. We predict how a social system works based on some belief about how humans will react with each other given the system’s institutional structure.

And yet, for an analysis of social systems, this is not enough. There are no moral statements to be made about the operation of the Darwinian system. Living things cooperate, free-ride, and devour or are devoured, and given the natural environment, some species flourish and some disappear. But to correctly observe that over the long term, single-celled organisms become multicellular organisms of ever greater complexity or some species emerge and others die out cannot be to make any moral claims about how the Darwinian system should operate. The biological system is constantly changing, and such change operates according to understandable principles. But unless one has a special place in one’s heart for the human species (as I do), this increasing complexity has no moral purpose; it is merely the empty one of propagating genes. No one can say that the global ecosystem of 100 million years ago was “better” than that of 50 million years ago or of today. That trilobites died out and were replaced by cephalopods, crustaceans, and other invertebrates of greater complexity is not a triumph, a tragedy, or a crime; it just is.7

If we are to judge human social systems, in contrast, moral criteria are essential. This book takes the utilitarian perspective, broadly defined. The purpose of a socioeconomic system is to promote a better way of life; the purpose of a scientific system is to promote the accumulation of scientific knowledge; the purpose of human communication is to promote more insight into the natural and human worlds. The concept of insight encompasses all claims that might be made by artists or moral or other philosophers through their work, as well as other insights that the accumulated commentary by readers or other “consumers,” including readers who are paid to render such judgments, might lend. To emphasize this purpose, unless discussing information traveling in only one direction, I refer to “communication” rather than “expression” or “speech,” emphasizing that the information flows usefully in multiple directions.

A System’s Components

Every social system has certain parts. The agents of a social system are those who participate in it. I choose this term specifically because of the philosophical idea of agency—the power to make things happen through one’s own choices. Second, a social system needs a goal. The three mentioned in the previous paragraph are the ones by which the systems in this book are judged. They are not further justified, even though it is possible to imagine alternatives. For example, one could judge a scientific system by the extent to which it preserves support for the existing religious hierarchy, or a socioeconomic system by the extent to which it generates equality in consumption. In theory, there may be multiple goals that can be in conflict or pursued together.

Agents will choose their goals based on the feedback they get, that is, information about the consequences that (may) occur from various choices. Feedback generates the incentives agents receive based on the rules of the system. While rules are often thought of as a set of what is and is not “allowable,” ultimately this is not a precise definition. To do something not “allowed” is usually not to violate the laws of physics but merely to court a set of negative consequences—fines, imprisonment, or worse in the case of a criminal justice system, for example. It will be very rare for the full set of consequences of the various choices available to an agent to be certain. But some feedback is more useful than others. If profits and losses, for example, are handed out randomly regardless of whether the product sold is high or low quality, or if criminal sanctions are handed out independent of whether the person receiving the sanctions committed the criminal act in question, then feedback is useless. We would be surprised if such a rule had any success in furthering the goal of eliciting socially valuable products or the avoidance of crimes. Alternatively, an omnipotent and omniscient system designer—the perfect “social planner” so beloved in much economics scholarship—would always impose whatever combination of feedback and incentives was required to achieve completely whatever goal that planner had. The “only” requirement for optimal system performance would then be that the planner’s goal was agreeable—that he be more like Plato or Buddha and less like Stalin. Presumably the incentives available to frail humanity are imperfect with regard to achieving the desired goals. Finally, networks refers to agents and the links among them in a self-regulating system.

Evaluating a System's Performance

The fact that we live in an all-too-flawed world means that social systems ought to be required to repair errors, or at least the consequences of those errors, once they are made. And so whether this system is homeostatic—in other words, whether deviations from choices that promote the goal are quickly corrected (not merely changed, but changed in a desirable direction)—is crucial in evaluating the system’s performance. A thermostat is a good example of a homeostatic system. If the temperature becomes too high or too low by some predetermined margin, a well-functioning thermostat will adjust so that the temperature returns to its programmed level. The goal of the homeostatic system is maintenance of a particular temperature range.

But errors are also addressed in other ways than periodic correction of deviation from some defined norm, a return to what I will call static equilibrium. Instead, the system can continuously improve. In science, flawed hypotheses can be abandoned and new ones adopted. In public contention among ideas about what society is and ought to be, mistaken views can be discarded. In an economy, the standard of living may rise as poor resource-use decisions are abandoned and better ones adopted. Such innovation results not just in new opportunities to use property in different ways for personal gain but for others to reap the corresponding benefits—better ways to promote health and more opportunities to travel and learn, for example. All of this will involve varying combinations of competition and cooperation among agents. This is not simply restoration of previous, optimal conditions (as in the thermostat setting), but dynamic progress in furthering the goal.

Effective error correction of either sort can be beset by difficulties, however. The success in achieving our postulated goal for the system as a whole may not be related to, or may be negatively related to, the particular goals of its individual members, who might give little thought to the performance of the system as a whole, as opposed to the achievement of their own goals. While many individual scientists care about scientific advancement writ large along with their own fame or career success, individual businesspeople generally do not think much about overall social progress when they make decisions about how to run their particular businesses. Participants in the public conversation on the great issues of the day, whether they are artists, journalists, or bloggers, probably do usually want to promote a better society, but they act primarily out of a conviction that what they say is true. They often mean to win the debates in which they engage—to persuade others that they possess Truth and that humanity is best served if they can get others to believe it. But the efficacy of the larger process by which we try to get at the truth is little on their mind.

In fact, to the extent that individual agents have incentives that conflict with the system’s overall goal, its error-correction attributes become all the more important. If a writer can make an extravagant, controversial claim that leads many people to agree with him when they buy and read the book he writes, his incentives to investigate carefully and thoroughly before making claims are correspondingly relatively diminished. If the advocate knows a claim is false but feedback incents him to make people think it is true (either because he believes some broader claim that this little white lie supports is true or because he has material or other interests furthered when many people think it is true), the costs of false claims are all the greater. Similarly, a business owner may not actively wish to sell a dangerous or otherwise inferior product, but if the monetary stake is large enough, he may persuade himself to neglect contrary evidence or even actively deceive potential buyers. So the error-detection process must effectively counteract these tendencies. To minimize the cost of such errors, the incentives should be structured so as to induce agents to do what, from society’s point of view, they ought to do, including making productive (i.e., instructive) mistakes.

This is, then, a condition for a self-regulating process to be effective. But political regulation of such processes, from a utilitarian perspective, also requires that two conditions be met. First, the political process must be able to effectively harvest knowledge about the problems generated by the operation of the self-regulating process. Second, political actors must on the whole use this information to craft solutions that will effectively address these deficiencies. It is important to remember these minimum requirements in comparing self-regulating and politically regulated processes.

A Guide to the Remainder of the Book

Fundamentally, this is a book about people figuring things out on their own. When they do, free of commands from outside the system, the results are often better. Whereas politics is rigid and monolithic, self-regulation is flexible and competitive. Self-regulation does not per se respect existing institutions, although people may use them in the course of creating new institutions and techniques. In a self-regulating system, there is a greater flow of things that might work, and things that don’t work are exposed and discarded more quickly. For a time from the seventeenth to the late nineteenth centuries, this thinking was consistently applied across a wide spectrum of human endeavor. The book documents this rise and (especially socioeconomically) its retreat. In doing so, it draws on several distinct scholarly traditions, especially study of the evolution of social institutions that bring out the best in us and mitigate our worst instincts in a variety of environments.8 The story begins with a discussion in Chapter 2 on the progress from hierarchical regimentation to the very idea of self-regulation. Humans are wired for competition as the occasion calls for, violent if need be, and most of human history is one in which might made right, and the mightiest thought it perfectly normal to regiment the rest of society. Chapter 2 tells this story, as well as of the rise in belief in self-regulating alternatives, based on the realization that some forms of competition can serve human purposes. The first comprehensive arguments for institutions based on self-regulation were made for science, as outlined in Chapter 3. Chapters 4 and 5 tell the story of the resulting development of similar arguments in the area of communication and socioeconomic activity. We will see that, especially in the latter case, arguments based on both discovered scientific principles and the operation of the scientific system itself were transferred to these spheres.

There are existing, albeit limited, versions of this type of analysis. For science, Michael Polanyi (1891–1976) explicitly used Adam Smith’s phrase “invisible hand” to describe the workings of the scientific process. He even argued that the rules of this process incent individual scientists to work on the “right” problem, given the goal of advancing scientific knowledge given individual scientists’ strengths and weaknesses. He also noted the importance of uncoordinated, dynamic self-regulation, although without using this term. Jonathan Rauch has also described the commonality of the two parts of what he calls “Liberal Science,” likening traditional scientific exploration to the search for ethical truth.9

Subsequent to the substitution of self-regulation for political regulation, the state of scientific knowledge, the span of human ideas, and our life possibilities all improved substantially. But self-regulation often requires tolerance of dramatic change, and such change is often resisted. So too it was with socioeconomic self-regulation in particular. Chapter 6 traces the development of the idea that socioeconomic self-regulation can malfunction. Chapter 7 describes the development of ideas about its fundamental flaws: that we should default, if not outright surrender, to political regulation as the proper form of organization for the socioeconomic system. The ideas in these two chapters amount to counterreactions to the claims of eighteenth-century Enlightenment thinking, which saw great moral purpose in lessening the grip of the state on individuals, who, when free to succeed and, critically, to fail were seen as the prime movers of social progress. Chapter 8 documents the growth of this counterreaction. In concluding, Chapter 9 offers some evidence on the negative effect of the partial retreat from self-regulation. It also speculates on the future of self-regulation in all three arenas and provides some advice about how to best take advantage of its extraordinary properties.


1. Baines. (1848), 40.

2. Velasquez-Manoff (2012).

3. See, for example, Pommerau (2008).

4. The economist Peter Boettke in a lecture once discussed the almost complete unfamiliarity of contemporary English speakers with the word ruth in its adjective form (as opposed to as a proper noun). Only when they hear the word ruthless can they infer this older, and largely vanished, meaning. Some political regulation of language does exist, with the Académie Française policing the use of loan words in French and the Chinese government introducing simplified characters in 1956 as two notable examples.

5. Yang (2013).

6. Morehouse (2015).

7. Today many argue that human-induced extinctions are immoral, in contrast to the many extinctions that took place before the emergence of the human species. The truth of that assertion is beyond the scope of this book

8. See the many examples in Ostrom (1990).

9. Polanyi (1962); Rauch (2014).