Introduction The science-politics paradox

Many of the world's most prosperous cities - including London, Hamburg and Rome - have within them a research centre studying what are called 'BSL4' organisms. These are dangerous organisms, which pose a 'high risk of life-threatening disease, aerosol-transmitted lab infections, or related agents with unknown risk of transmission'. The Labs that study them are also labelled as BSL4. They are familiar from the evening news as places full of people wearing what look like space suits, and they often show up in Hollywood films.

It's estimated that there are some 69 of these either in operation or construction worldwide. Most are located in urban areas. Most of the people living nearby, which probably includes many readers of this sentence, are unaware of them. Most might be shocked if they knew what was being done within them, often with only weak oversight and regulation,1 and would wonder why they are located in highly populated cities, like the one in Wuhan which some thought might lie behind the COVID-19 outbreak.2

Much of their work is necessary. But they are making judgements about risk that have huge significance for everyone else, for example when they experiment with combining viruses, or amplifying their harm or transmissibility. Yet there are no global agreements on how they should be run, no official registries of where they are, what they do, or how safe they are.

This is one of many examples of a gulf that has grown up between science, which is often necessary and inspiring but also often opaque and secretive, and public interest or public dialogue, or what some would call common sense. They are examples of the challenge every society has of exercising power over knowledge and reminders that while science may save us, whether from diseases and natural threats like asteroids, it may also kill us, whether with a nuclear Armageddon, invented pathogens, or wayward intelligence.

The ambiguities of science and technology are evident in the impacts of the many technologies that have seen exponential improvements in recent years. The processing power of computers has roughly doubled every two years in line with Moore's law. The cost of sequencing the full human genome fell from around $100 million two decades ago to $100 today, while the cost of solar powered electricity fell almost 90 per cent in the 2010s alone.3

These advances happened alongside stagnant incomes for many in countries such as the US, stagnant wellbeing, declines in social connectedness and mental health, as well as worsening global ecological indicators and signs of potential systems collapse.

Young people reflect these paradoxes: most are very positive about science and technology, more than their parents' generation, and technology plays a big part in their lives. According to one survey of 20,000 young people across the world, 84 per cent of them say that technical advancements make them hopeful for the future. But this enthusiasm for science combines with pessimism. In sixteen out of twenty countries, more young people believed the world was becoming a worse place to live than believed it was becoming better, with many of the reasons cited being indirect effects of science, from workplace automation and climate change to the harmful effects of social media.

Science is the most extraordinary collective achievement of the human species - a set of methods, mindsets, theories and discoveries that have changed every part of our lives. But these paradoxical patterns show that a powerful method for amplifying human intelligence is not always so intelligently directed.

So you might expect that the question of how to govern science, how to mobilize its benefits but avoid its risks, not just in biohazard labs but also in everything from artificial intelligence and food systems to space warfare, would be one of the most important questions of our times.

But how? Scientists have long argued that they should be given the maximum freedom to explore and discover. While there are good arguments for this, particularly in more fundamental science, it becomes ever less plausible the closer science and technology come to daily life. Although scientists are typically intelligent, thoughtful and decent people, it's not obvious that they can be trusted to govern science, any more than the military can be put in charge of wars. Science alone can be tunnel-visioned: it needs other perspectives to show how to avoid harm.

This is where institutions come in. Our societies are made predictable and manageable through institutions. It's through institutions that a public interest comes to be refracted. Within nations, an array of funders, regulators, agencies, commissions and parliamentary groups try to steer science and innovation. But there are glaring governance gaps - gaps where institutions are needed, whether in relation to AI and cybersecurity or synthetic biology. Moreover, public influence over science has declined in recent years as the proportion of R&D dominated by big firms has grown. Amazon, for example, spent $40bn on R&D in 2020, more than all but a handful of countries (the UK's public R&D budget that year was around $14bn, Finland's around $2.3bn).4

These gaps are even more evident at a global level where there is little effective governance. We have a World Bank, an International Monetary Fund and numerous other funds and development agencies. But the world lacks institutions charged with reflecting on science, its aims and its methods, or judging whether the world's research capacity is directed to the right tasks and with the right methods. Scientists have achieved many breakthroughs in global cooperation, often below the radar of formal politics - from the rules of the Internet to the management of Antarctica and nuclear non-proliferation. Yet the best available data show a striking lack of alignment between what the world has decided are its top priorities - summarized in the Sustainable Development Goals - and the priorities of science and technology.5

Scientists are periodically involved in public calls to guide, constrain or rein in powerful technologies, particularly artificial intelligence. But, as I show later, these are usually so vague, and so devoid of any plan of action or any language for thinking about governance, that they have little impact.

In the nineteenth century, as constitutional monarchy became the norm in much of Europe, it was said that monarchs now reigned, but didn't govern. Science is now in an opposite position. It governs but doesn't reign and is only loosely accountable for the power it exercises.

Some of the reasons for this lie in the blind spots and biases of science itself. Scientists say they can't make policy, that they have neither the skills nor the inclination to do so, and they fear being tarnished if they get too close to the grubby, compromised world of actual government. Yet their de facto power makes this position of detachment increasingly implausible.

Politics should be the answer, since it is the main way we make collective decisions. But politics looks ill-suited to the task of governing science. The dominant forms of modern politics were shaped in the nineteenth century: rule by representatives concentrated in parliaments in capital cities, with periodic elections, manifestos and programmes. There has been relatively little advance since then, despite many experiments on the periphery (from citizen assemblies and deliberations to virtual parliaments). Instead, politics often looks petty, short-term, half-informed, or irrelevant. The ways in which politicians are recruited and promoted don't fit well with the tasks they have to fulfil and their roles are almost unique in being so unsupported by professional training - most learn on the job.

I've sometimes played a slightly mean trick on senior figures in politics.6 I ask them if they could give a five-minute talk on how the Internet works. Almost none can, though they use it for many hours a day. They know next to nothing about how it functions, or about the material reality of undersea cables and switches or the organization of addresses and protocols.7 Essentially for them it is magic, which perhaps helps explain why governments and parliaments found it so hard to respond intelligently as the Internet transformed so many areas of life, both for better and for worse.

So, for politics to play the roles that only politics can play we need a radically reformed politics. This is what I call the 'science-politics paradox': only politics can govern and guide science in the public interest, but politics has to change to be able to do this: to become more knowledgeable, more systematic in its methods, and some of the time, more scientific, benefiting from what I call the 'new curriculum for power' that encompasses data and systems, complexity and psychology as well as politicians' more traditional grounding in law and economics.

I use Hegel's story of the master and the servant as a way to make sense of this dynamic. Politics, the putative master, has nurtured a servant who now greatly outstrips the master in terms of capability and knowledge. Science has gained a de facto sovereignty of its own, that sits along the traditional sovereignty of politics: the servant has to some extent become a master. Most of our collective decisions now involve science - from pandemics to...