Tracing the dynamics of progress in time and space reveals a fundamental tension running through the historical record: technologies that strengthen bureaucratic control can temporarily spur development but later hinder new discoveries and sow the seeds for stagnation.
Early dynastic China leveraged technology to achieve its goals, such as tax collection, power consolidation and warfare. The construction of the Grand Canal, along with the development of thousands of irrigation and flood control initiatives nationwide, testifies to the ability of the Song dynasty to drive technological progress from above.
Over the long run, however, early centralisation left many technological trajectories unexplored. According to Joseph Needham’s seminal work, China relinquished its position as the global leader in innovation by the 14th century, and subsequently, no major breakthroughs were made in the Middle Kingdom.
Ultimately, the drive to establish an impersonal and meritocratic system of bureaucratic recruitment led to the suppression of grassroots innovation and diversity of thought. This created political stability but also serious obstacles to reform and continued progress. Skewed incentives encouraged China’s most able youths to pursue a career within the government bureaucracy, which significantly weakened its science and technology sector.
Worse still, during the Qing dynasty, the Chinese state was eventually captured by vested interest groups. Overburdened magistrates depended heavily on local gentry and merchant guilds, which strongly resisted tax increases. A clear example is 19th-century Xinhui, where a powerful guild prevented taxation by blocking surveys of taxable areas. Consequently, the government increasingly granted monopoly rights to kinship groups, merchants and guilds in exchange for informal revenue, a practice that likely impeded long-term innovation and economic progress. As a result, China was overtaken by a more competitive Europe, where intellectuals and inventors could escape the heavy hand of the state and eventually guilded regulation.
At the technological frontier, continued progress depends on the power and willingness of governments to override vested interests to restore competition. The birth of the Industrial Revolution illustrates this vividly. The revolution first gained momentum in Britain, where inventors such as James Watt and Richard Arkwright turned their ideas into practical applications and gave rise to the factory system that would fuel the global economy for centuries to come.
This was in stark contrast to the relatively successful resistance encountered by industrialisation in other parts of Europe, where powerful craft guilds saw their interests threatened by this new economic order. In Britain, however, decentralised competition between cities and the early integration of markets helped to undercut the influence of guilds and to unleash the forces of creative destruction. But Parliament’s supreme authority also played a crucial role, because it insulated politics from local interests.
Compared with England, the Dutch Republic, often hailed as the first modern economy, was relatively democratic. This meant, however, that its political institutions were more susceptible to being captured by incumbents and special interests, who often sought to impede technological change. The core issue was that, alongside a central assembly for the entire republic, each Dutch town and province maintained its own local assembly. And for any proposal to pass in the higher-level assemblies, it needed approval from every major town. This created gridlock, as powerful interest groups found ways to manipulate trade regulations to their advantage. Over time, as local merchant guilds grew stronger, they used their control to block competition and innovation.
In France, too, the capacity of the government to check vested interests proved critical. Indeed, it was only with the French Revolution and the crowning of Napoleon that the old order was swept away, the guilds were abolished and a centralised state capable of promoting competition was established and spread its ideas throughout Western Europe. In the Rhineland, where guilds had successfully resisted technological change, the introduction of the French civil code undercut the influence of German local elites, which resulted in the disbandment of guilds and a surge in innovation.
At the dawn of the Industrial Revolution, what truly set Britain apart was the relative lack of power afforded to incumbent economic interests. Unlike in the Dutch Republic, British Members of Parliament were not beholden to strict mandates from their constituents. Instead, they were able to legislate as they saw fit. Britain’s industrial revolution thrived not only because Parliament upheld private property rights but also because it was prepared to set them aside when the nation’s growth demanded it. The construction of canals and turnpikes, for example, was driven by Parliament’s use of private Acts, allowing public infrastructure projects to move forward despite resistance from individual landowners.
America likewise benefited from both the lack of and ability to overcome incumbent interests. As an infant republic, the United States did not have to battle with the legacies of the old regime, and for most of the 19th century, inventors did not have to worry about the kind of gilded interests that prevailed on the European continent, where they blocked many technological pathways from being explored. It was, for instance, the absence of gilded restrictions that made Heinrich Steinweg leave his native Germany to set up Steinway & Sons in New York City. True, the expansion of the frontier of free land undoubtedly was a key magnet for much talent, but other nations such as Russia, Argentina and Chile experienced similar periods of territorial expansion, yet failed to achieve the same level of inventive success as the United States.
The key difference lay in the institutions of each country. In America, the Constitution created a framework of private property rights and free trade, which provided a strong foundation for innovation and entrepreneurship, attracting talent from across the globe. The filing fee for obtaining a patent in the United States was, for example, only a fifth of the price charged in Britain, which enabled a much larger cross-section of the population to participate in innovation. It was the work of lone inventors such as Samuel Morse, and not corporations like Western Union, that paved the way for the development of the telegraph. And Morse and others like him were supported by a liquid market for technology that allowed inventors to specialise in exploration.
That the Computer Revolution happened in the Bay Area, of all places, is again no mystery. Like Britain’s Birmingham and Manchester, which sprang up in formerly rural areas free from guild regulations and became the engines of the Industrial Revolution, Silicon Valley’s lack of a prior industrial history and disconnect from established economic and political structures fostered an environment conducive to innovation.
What the Valley’s history shows is that a competitive and decentralised system is key to discovery, especially when barriers to horizontal movement are absent. Silicon Valley’s rise is attributable in no small part to a culture of job hopping that can be traced back to the California Civil Code of 1872, which outlawed covenants in employment contracts. Nearly a century later, this law would permit the ‘traitorous eight’ to break away from Shockley Semiconductor Laboratory and set up new companies, Fairchild included. So, despite the existence of incumbent, centralised technology clusters in Boston and Detroit, America’s federal system created the space for continued experimentation. By allowing for variations in policy and governance at the state and local levels, federalism provided an institutional laboratory, which proved crucial to America’s continued progress.
This is an edited extract from ‘How Progress Ends: Technology, Innovation, and the Fate of Nations’ (2025) by Carl Benedikt Frey published by Princeton University Press and reprinted here by permission.
