Why care about institutional design
How institutions amplify human nature, for good and ill
May 14, 2026
Atrocity at scale is an institutional achievement, not an institutional failure. The Holocaust needed census lists, railway timetables and career incentives; Rwanda in 1994 needed radio, functioning local councils and an administrative hierarchy. The coordinating machinery that lets a million strangers cooperate on a railway network is the same machinery that lets them cooperate on a deportation, and it does not know the difference.
That is the case for caring about institutional design, and it is historical rather than philosophical. Institutions are the multiplier that selects which face of human nature mobilises at scale: the same population under different substrates produces wildly different collective behaviour. We are now setting the substrate inside which AI develops.
1. Two faces of human nature
The historical record is full of suffering — slavery, conquest, massacre, ritual cruelty, sexual coercion, mob violence — across continents and across centuries. Keeley (War Before Civilization, 1996) assembles violent-death rates across non-state societies that run from low single figures to nearly 60% of adult male deaths, depending on the group and on whether the metric is deaths from war or deaths of adult men; the ~15% often quoted is Pinker’s later synthesis rather than a figure Keeley states. These numbers are disputed, and seriously. Ferguson (“Pinker’s List”, in Fry ed., War, Peace, and Human Nature, 2013) argues the archaeological sample is selected toward fortified and contested sites and that skeletal injuries are routinely miscoded as warfare deaths; Fry and Söderberg (Science, 2013) make a separate ethnographic case, finding that the mobile forager bands closest to the ancestral condition show markedly lower rates. Lower estimates are defensible. It does not matter much for the argument: the claim needed here is only that predation is a robust mode of the species, not that it was the dominant one, and genocide’s recurrence across every continent and era establishes that much on its own.
The same record is full of cooperation. Humans are hyper-prosocial relative to our primate kin — cooperative breeding (Hrdy), strong reciprocity and willingness to punish defectors at personal cost (Bowles & Gintis), public-goods-game cooperation rates far above what rational-actor models predict. Strangers help strangers, large groups coordinate around shared goods, individuals sacrifice for groups they will never personally meet. Cooperation is not a moral varnish over a violent base; it is also a real mode of the species, also robust.
Human nature is plastic — both modes are available, both are deep. The interesting question is not which one is real; both are. It is what determines which one mobilizes at scale.
2. Institutions as the multiplier
Institutions are the answer. They are the scaffolding within which large numbers of individual humans coordinate, and the scaffolding determines whether the coordination expresses the cooperative mode or the predatory one. The same logic operates across very different applied settings — state form, regulatory agency, corporate charter, professional norm, international treaty.
Cooperation amplified. Tax-and-redistribution coordinates a latent willingness to help strangers — most people are willing to contribute to those worse off, but cannot do so unilaterally with any confidence the contribution lands. Courts and due process replace private revenge with adjudication, turning vendetta into procedure. Environmental and workplace-safety regulation price externalities that the unregulated market has every incentive to keep externalising. Limited liability and contract law make trust portable, converting one-shot interactions between strangers into repeated games with consequences.
The digital cases are the cleanest, because the platform holds everything else fixed and varies only the rules. Wikipedia produces a usable encyclopedia out of arbitrary internet strangers using edit history, reverts, talk-page deliberation and explicit dispute resolution. Stack Overflow extracts high-quality technical answers from a global pool using reputation, voting and strict moderation. r/AskHistorians produces expert discourse on the same underlying platform whose default settings produce mob behaviour. Same humans, same substrate, different rules, different species.
None of these institutions creates the cooperative impulse. Each one organizes it.
Predation amplified. Here is the claim that matters most in this post, and it runs against the intuition most people bring to the subject.
Atrocity at scale is an institutional achievement, not an institutional failure.
Nazi Germany did not release a suppressed primal hatred. It organized hatred at industrial scale — census lists, bureaucracy, railway timetables, a propaganda apparatus, professional administration, career incentives. Rwanda in 1994 was hyper-institutional: radio broadcasts naming targets, functioning local councils, and killing directed down the commune–sector–cellule hierarchy under serving officials. (How much work the broadcasts did is contested — Straus argues face-to-face mobilisation at barriers mattered more.) The Atlantic slave trade required shipping insurance, plantation law, financial instruments and sovereign treaties; it was one of the most sophisticated logistical enterprises of its century. Apartheid was a legal code. The Inquisition was a bureaucracy.
None of these was a collapse of scaffolding. Each was scaffolding, competently built, pointed at a different target. The coordinating machinery that lets a million strangers cooperate on a railway network is the same machinery that lets them cooperate on a deportation, and it does not know the difference.
The weaker version is predation permitted rather than organised — the regulatory-absence case, which §3 takes up with the evidence attached. It needs no malice, only the absence of a rule internalising an externality.
The contemporary corporate and digital versions are recognisably the same object. Purdue Pharma organised addiction at scale through a sales force, pain-management protocols, distribution channels and regulatory capture. Tobacco and fossil-fuel disinformation needed research labs, PR firms and lobbying machinery to manufacture decades of doubt. Engagement-maximising recommendation algorithms amplify outrage by design; Facebook’s feed has been documented contributing to ethnic violence in Myanmar, and WhatsApp groups in India tied to mob lynchings. 4chan gets harassment and coordinated manipulation from anonymity, no persistent reputation and minimal moderation — the same human beings who behave differently elsewhere.
Predation at scale is rarely what happens when institutions fail. It is what some institutions actively do, and what others actively fail to prevent.
The unified design space. These institutions sit in very different applied settings — sovereign state, federal agency, joint-stock company, profession, treaty body — but the abstract object underneath them is the same: a set of rules determining payoffs and information flow among many self-interested agents. Game theory, mechanism design, multi-agent reinforcement learning, social choice, public economics — these are the formal apparatuses studying the same thing across different applied surfaces. The design space is unified at the level of theory even when the application is not, and the rest of this series works that unified space.
The same coordinating power that organizes cooperation can organize atrocity. Institutions are not neutral conduits for human nature; they are the multiplier that selects between its modes.
3. The natural experiments
The hypothesis is empirically testable, because history has occasionally produced controlled comparisons. Two kinds: same people, different rules in different places (cross-sectional), and same people, different rules over time (temporal). Both confirm the §2 picture — when the rule changes, the mode of human behaviour that mobilizes at scale changes, and the welfare consequences are large.
Cross-sectional: same people, different polities.
The Korean peninsula. In 1945 Korea was a single ethnically and culturally homogeneous population, divided arbitrarily at the 38th parallel by the US and USSR; three years later the line hardened into two states. The two halves received sharply different institutional substrates — one eventually broadening participation, electoral, with enforced property and contract; the other concentrating extraction by party rank. Eighty years on, South Korea’s per-capita income is roughly thirty times North Korea’s on the Bank of Korea’s estimates — a figure worth treating loosely, since North Korean output is inferred indirectly and other sources put the ratio higher. Life expectancy, infant mortality, literacy, infrastructure quality — every measurable indicator shows the same divergence. Same people, same geography. Different scaffolding around the same human-nature baseline.
Nogales. The city of Nogales sits across the US–Mexico border. The same families live on both sides; cultural and geographical conditions are essentially identical. The institutional environments are not — different legal systems, different enforcement of property rights, different access to public goods, different exposure to extractive informal power. Average household income differs by roughly a factor of three across the same town. The case is suggestive rather than clean: the two Nogaleses also differ in migration selection, remittance flows and exposure to the drug war, none of which the comparison controls for.
Colonial origins. The Korea and Nogales comparisons have an obvious weakness, and it is worth stating before someone else does: institutions are not randomly assigned. Whatever caused a place to end up with extractive institutions may also have independently caused it to be poor. Anecdotal natural experiments cannot separate the two.
Acemoglu, Johnson and Robinson (2001) exists precisely because of this problem, and its contribution is the instrument, not the correlation. They use settler mortality in colonised territories as an instrument for institutional quality: where Europeans could survive, they built inclusive settler institutions; where disease killed them, they built extractive ones. Settler mortality plausibly affects modern income only through the institutions it selected, which is what makes the estimate causal rather than correlational.
The instrument is contested. Albouy (2012) argues the mortality data are unreliable — that many rates are extrapolated from other regions, that campaign and epidemic conditions are conflated with settled ones, and that the results are sensitive to which observations are included. The exclusion restriction has also been questioned, since disease environment plausibly affects modern productivity directly. The honest summary is that AJR is the best identification anyone has produced for this claim and that it is not settled. Institutions-matter survives on the weight of many imperfect sources rather than on one clean one.
Temporal: same population, different rules over time.
The Thames. Until the 1860s London emptied raw sewage directly into its drinking-water source, and successive cholera epidemics killed tens of thousands. Bazalgette’s interceptor sewers, begun in 1859 and substantially complete by the mid-1870s, ended waterborne cholera in the city — though not overnight, and not alone: the 1866 East London outbreak killed some 5,600 people in the one district not yet connected, and the Metropolis Water Act 1852 deserves a share of the credit. The change was not in human nature, in geography, or in technology in the deep sense; it was in a regulatory regime that finally internalised an externality the existing economic structure had every incentive to keep externalising.
Air quality. The 1952 Great Smog of London killed around 4,000 people in five days, mostly from acute respiratory failure; later epidemiological work (Bell and Davis, 2001) puts excess deaths through the following winter at roughly 12,000. The Clean Air Act 1956 introduced smoke-control zones that progressively displaced domestic coal-burning, with further controls in 1968. The atmosphere over a contemporary industrial city is dramatically less lethal than the same atmosphere a century ago.
Financial stability. Between 1873 and 1933 the US experienced major banking panics roughly every decade — 1873, 1884, 1890, 1893, 1907, and the cascade of 1930–33 — destroying the savings of millions of ordinary depositors. The 1933 Banking Act (FDIC + Glass-Steagall) introduced deposit insurance and separated commercial from investment banking. The US then went roughly fifty years without a bank run of the old kind. Whether the post-1999 repeal of Glass-Steagall’s separation contributed to 2008 is genuinely contested — the firms at the centre of the crisis were largely not the combinations it had barred — so this is a suggestive reverse experiment rather than a clean one.
These cases are selected on their outcome, and §4 supplies the corrective: deliberate design fails often, and the successes here share a narrow and specific shape. Read with that caveat, what they have in common is the variable: the rule — who gets to do what, who is liable for what, what claims are enforceable, what externalities are priced, what concentrations of power are tolerated. Change the rule, and the mode of human behaviour that mobilizes at scale changes with it. Which face of human nature got mobilized dominates which humans were available to mobilize, and the rules are what determine which face.
4. Mixed cases
The picture would be too clean if institutions only ever ran in one direction. They don’t.
The Roman Empire institutionalized law, contract, citizenship, and infrastructure — and ran on conquest and chattel slavery. The British constitutional settlement after 1688 produced parliamentary government, secure property rights, and the Industrial Revolution — and underwrote the largest extractive empire in history. The antebellum United States combined constitutional democracy for some with chattel slavery for others, in the same legal system. Institutions amplify cooperation among the included and predation toward the excluded. The boundary of the inclusion is often the most consequential institutional choice.
This matters for the series because it means “institutions matter” is not the same as “more institution is better”. Bad institutional design is design — it produces effects, sometimes catastrophic ones, with the same coordinating power that good design uses for opposite ends. The problem is not under-institutionalisation; it is the design of the institutions that exist.
Deliberate design that failed. Every case in §3 is a rule change that worked, which should make you suspicious of the sample. Four that did not:
Prohibition. The Eighteenth Amendment and the Volstead Act were a deliberate, constitutionally-enacted institutional intervention against a well-identified harm. They produced large-scale organised crime, mass non-compliance, and a durable corruption of enforcement; national prohibition took effect in January 1920 and was repealed just under fourteen years later. The design was serious; the model of how people would respond to it was wrong.
Post-Soviet shock therapy. Rapid privatisation and price liberalisation were designed by competent economists working from an explicit theory. In Russia the result was the transfer of state assets to a small group under conditions of weak property enforcement, a collapse in output, and a fall in male life expectancy of several years. The institutions arrived in the wrong order: markets were installed before the legal and administrative capacity that makes markets work.
IMF structural adjustment. Conditionality programs across Africa and Latin America in the 1980s and 90s applied a common institutional template across very different starting conditions, with results that are at best mixed and in several cases clearly negative.
The DAO. A governance mechanism designed from first principles, formally specified, and deployed with real money. It was drained of roughly $50M in 2016 through a reentrancy bug in the code that constituted it, and the community’s response — a hard fork — was precisely the discretionary human override the design existed to eliminate. This one is the most instructive for the series, because it is the closest thing we have to designing an aggregation mechanism from a specification, which is what the rest of these posts propose doing.
What the failures have in common is not that the designers were foolish. It is that they were confident about how people would respond. That confidence is the recurring error, and the case that it is never warranted is the subject of the human-nature companion.
5. When institutions fail, preferences themselves shift
The hardest part of the story is that the substrate doesn’t only channel what people do; it shapes what people want.
Weimar Germany is the canonical case. Hyperinflation, depression, Versailles humiliation, electoral fragmentation, and political violence in the streets produced a population willing to vote for and accept what would previously have been unthinkable. The standard explanation is multi-causal — desperation alone did not do the work; it lowered resistance, and a competent organizational machine supplied the rest. Mobilising resentment takes entrepreneurs and infrastructure, not just psychic pressure. Syria after 2011 is the contemporary analogue: drought (whose causal role is disputed), regime brutality, sectarian fault lines, regional proxy dynamics, and a collapsing institutional substrate produced a population whose available political choices narrowed to versions of the predatory mode. Material desperation is not by itself a moral solvent; it is the precondition on which political entrepreneurs construct a preference distribution that the prior institutional substrate had been suppressing.
Wealthy democracies are not immune. The contemporary drift toward populism, polarization, and authoritarian sympathy in societies that are objectively prosperous shows that absolute material conditions alone don’t fix preferences either — perceived status loss, cultural displacement, and organized resentment do their own work. The institutional substrate has to actively form preferences toward the cooperative mode; absent that work, the predatory mode is always available to be mobilized.
This is the deep version of the institutional argument: the preference distribution that any mechanism has to aggregate is itself a product of the institutional substrate that came before. Bad institutions don’t just fail to do their job. They actively break the inputs to the next round.
6. The AI institutional choice
We are now setting the institutional substrate inside which frontier AI develops — the rules governing labs, the corporate forms they sit in, the regulatory environment around them, the procedures by which models acquire their objectives. These rules are being established right now, and path-dependence will lock them in for a long time. Several decisions are visible:
- Whether AI labs are governed by shareholder primacy or by a broader stakeholder structure.
- Whether model weights and training data are owned by the labs that produce them, or are subject to public-interest constraints.
- Whether the procedures that determine model objectives (RLHF, Constitutional AI) include input from outside lab leadership.
- Whether compute provision is structurally separable from model development.
- Whether the regulatory bodies overseeing AI are independent or industry-aligned.
- Whether AI labs are subject to liability for systemic harms produced by their systems.
These are assertions, not findings, and it is worth being explicit about which of them the historical record actually speaks to. The record is strong on liability (the FDA, OSHA and Clean Air Act cases are all fundamentally about who bears the cost of a harm) and strong on regulatory independence versus capture (§3’s financial-stability case, and the tobacco and opioid cases in §2). It is moderate on ownership structure, where the corporate-form evidence is real but confounded. It is close to silent on who provides input to the objective-setting procedure, which is the item this series cares most about — there is no historical case of a large institution deliberately broadening the input set to its own goal-specification procedure and a measured result. That absence is itself informative, and it is the gap worth working in.
Each of these is a rule (or mechanism) in the sense the rest of this series uses the word. The combination determines the implicit objective of frontier AI development, in the same way that the combination of shareholder primacy, limited liability, fiduciary duty, and board structure determines the implicit objective of the modern corporation. The historical record suggests this choice will dominate almost everything else we might worry about — choice of training algorithm, capability evaluation method, interpretability technique. Pursuit machinery — the question of whether an optimiser actually chases the goal it was given — is necessary but downstream. The institutional substrate is upstream.
7. What we don’t yet know
The gap between what this post shows and what the series needs. Everything above establishes that institutions matter — that the same population under different rules produces radically different outcomes. The series needs more: that we can deliberately move institutions in a chosen direction. Look at what the successes in §3 actually are. Bazalgette’s sewers, the 1938 Food Drug and Cosmetic Act, workplace-safety law, the Clean Air Act, deposit insurance — every one is a narrow, single-externality intervention with a clear causal model and a measurable target. Not one is an example of designing an aggregation mechanism from a specification of what it should aggregate toward.
That is a real gap, and it is Hayek’s opening. His claim is not that institutions don’t matter but that the knowledge required to design them deliberately is dispersed and tacit, so that what works is evolved and selected rather than specified. The evidence assembled here is compatible with his view. Narrow interventions against identified externalities are exactly the cases where a designer can have the required knowledge; the absence of successful designed aggregation mechanisms is exactly what he would predict.
I don’t think this sinks the program, but it sets the burden: the series has to show that designed aggregation is possible, not assume it from the fact that plumbing regulation worked.
We do not have a rigorous theory of institutional design. We have empirical regularities, a handful of formal results from social choice and mechanism design, and a long historical record. Much of what actually makes good institutions work is informal — social norms against lying and misleading, expectations of good-faith compromise, professional restraint, dense webs of trust between people who have to keep working with each other. We can name these but we cannot yet specify them, design for them, or repair them when they erode. The rest of this series is an attempt to take what we do know — the formal apparatus of social choice and mechanism design — and push it toward something that could eventually inform the AI institutional choice.
The stakes are sharper than the historical analogues suggest. In the near future, an institution will be in charge of AGI — its training, its deployment, its objectives, its constraints. Whether that institution is a corporate board, a national government, a treaty body, a DAO or some diffuse combination of all, it will effectively act as principal for a system capable of consequential action on behalf of — or at the expense of — the rest of humanity. The track record of who has been principal for the previous most-consequential systems humans have built — the modern corporation, the nation-state, the central bank, the regulatory agency — is not reassuring. We have been bad at this for systems with much smaller blast radius. AGI is the moment when we cannot afford to be bad at it.
So take this post as establishing the weaker of two claims. It shows that institutions matter enormously; what the rest of the work needs is that institutions are deliberately designable, and the evidence here does not get there. That gap is where the serious objections live, and the procedural framework assembles what follows if you are willing to bet across it.
For the empirical case: Acemoglu and Robinson’s Why Nations Fail (2012) and The Narrow Corridor (2019); North, Wallis and Weingast’s Violence and Social Orders (2009) for the abstract framework; Bowles and Gintis’s A Cooperative Species (2011) for the prosocial side; Ostrom’s Governing the Commons (1990) for self-organised institutions that no one designed. For the other side of the disputes flagged above: Albouy (2012) on the settler-mortality instrument, and Fry and Söderberg (2013) against the Keeley/Pinker violence estimates. For the case against the whole enterprise, Hayek’s Law, Legislation and Liberty.
This post is part of a series on governance mechanisms.