This season’s ‘in’-word for policy wonks is ‘Superforecasting’. It is the science, or rather social science, behind trying to nail down the percentage likelihood of a given event happening.
What we are witnessing now shows just how important getting this kind of thing right. But when the dust eventually settles, rather than reflect on simply the question of Prediction, policy makers should instead take a more ambitious plunge and delve into the fields of Risk Management and Resilience. An encompassing strategic review is well overdue.
Unhelpfully, the brand of the superforecaster has been latterly tarnished by the sacking of a SpAd who self-identified as one (though his actual credentials are unclear). This should not distract us from the key lesson that until very recently there has been a shortfall across political advisers in understanding deep risk.
The key book on Superforecasting is the eponymous volume by Philip Tetlock and Dan Gardner. As a result of extensive analysis and testing, their research identified methods that can improve prediction making, and therefore might by extension inform risk taking.
But there are still key gaps in their work. One problem is that superforecasters’ output and relevance are overwhelmingly dependent on the choice and parameters of a binary question, one set by someone else. Methodology is so diverse that some even deliberately choose to not share their working assumptions on the basis it might affect the mean estimate, ruining the ‘wisdom of crowds’ averaging-out approach. In any case, as non-Subject Matter Experts (SMEs), they are utterly dependent on expertise of others; indeed, some of the most successful superforecasters gain that rank precisely because they chase after informed opinion more persistently than their counterparts.
So Superforecasting is an adjunct. It would correspondingly seem the optimal solution is not to test thousands of people to determine who is the best go-to Vulcan for oracular advice, but to train the professional analysts to better frame the problem, to disentangle the root variables, to glance more at indicative windsocks, and to repeatedly challenge assumptions.
Logically, that might at times mean combining a team of experts with an outsider brought in as a safeguard check – a version of the ‘Tenth Man Rule’, made famous in the film World War Z but actually a Red Cell principle introduced by the Israelis after the Yom Kippur War.
It’s also worth contrasting the Superforecasting approach with that set out by Nassim Nicholas Taleb, who introduced the celebrated Black Swan principle to the world – the idea of flawed analysis that fails to factor in ‘unknown unknowns’ and which takes recent lack of surprises as proof of absence of risk. That’s the sort of business approach that stoked the sub-prime mortgage crisis, not to mention the Tulip Bubble, the South Sea Bubble, and the Mississippi Bubble.
Whereas Superforecasting suggests you can apply logic to call better odds on a specific event taking place, Taleb’s countervailing observation is that while you can spot people making bad assumptions based on false constants, nevertheless you cannot properly predict the rare wild cards. Superforecasters are therefore as bad as everyone else in missing the ‘fat tail’ big events.
No doubt supporters of both sides will rebuke me as oversimplifying. The key point is that having superb forecasters alone, even if well directed, isn’t enough. Democracies tend to be short-termist in their strategies, and even shorter in their funding plans. That works well until it doesn’t. Both policy drafters and especially those advising the guardians of the purse strings need to be better informed on what relative value any spending on insurance policies may bring. Cost-Benefit thinking needs to be strategically appraised by politicians over a longer term.
It might mean, even if our current problems do fade away, that more funds are needed to prepare for the next novel virus, including stockpiling greater physical resources in some latterday RAF Burtonwood. In medical terms, again it may be about ensuring that the UK simply has the core facilities for a future conflict should we be engaged in something bigger than a peacekeeping or support mission. The recent MoD bid, championed by Penny Mordaunt, for DfID money to fund a mediship now seems like a wasted opportunity that we should revisit. I wonder how many people will shortly be begrudging the Queen in retrospect the use of HMS Britannia, given its stand-by medical role.
Equally importantly, how prepared are the nuclear states for avoiding an accidental conflict? I’m not making the case for disarmament, given the genie and the bottle, but policymakers do need to understand the history of an alarming number of near misses we have had, and where a human being has been put on the spot to make a decision based on what turned out to have been false readings, and could have made a different call.
What about prep for a surge in solar flare activity and what that will mean for everything that depends on satellites? The US military has done work to protect its electronics from frying during a Carrington Event, but what about everyone else?
For those operating on a lower special effects budget, what about the appropriateness of the contingency planning around an aggressive trade war? We still have North Sea oil on tap, but what of the knock-on impacts of Russian gas being switched off for Western Europe as it has for Eastern? How quickly as a nation we have forgotten the Three Day Week. What of stockpiles of rare earths, or even basic metals? What too of strategic food stocks, or of processed fuel reserves?
What about the UK military’s ammunition reserves? Or the reserves of high-tech military equipment that is slow to replace, and whether we are capable of even plugging existing shortfalls in time? In many of these cases, considered review will reveal trusted suppliers – and the added incentive in terms of policy direction of developing our post-Brexit alliances with our CANZUK brethren.
Into this mix we must throw economic policy. Are our gold reserves sufficient, such as they are post-Gordon Brown? What of policy over our national debt, as well as that complex double-edged sword of who holds it? The open pit of subsidising potentially the entire economy for weeks or months during any virus shut down is only sustainable if you haven’t run a longstanding deficit leaving you carrying debt like Atlas before you start.
This is just a short list of some very big potential crises, of variable risk and unknown imminence, though some are cyclical. No doubt the relevant parts of Whitehall have over the years made considered and detailed contingency plans covering every case. But whether they deserve more funding is where the politics lies. Uninformed policymakers make for bad policy, or none, and there is an inherent disincentive for any “here today, gone tomorrow” politician to dedicate funding. Even to ask the question and explore the odds invites ridicule.
At least the Swedes have seen some value in publicly pushing risk management, and I suspect are seeing some benefits right now. Even though Stockholm considers the prospect of conflict with its near neighbours to be low, the Swedish Government in 2018 distributed a 20 page pamphlet “If Crisis or War Comes”, giving every household advanced planning time to get ready in case the country ever went offline or shut down for whatever reason.
It also incidentally would have sent a message to anyone considering engineering such an asymmetric attack, thereby reducing not just the consequences but the prospect. The same precaution in a dangerous world underwrites the need for the UK to spend 3% of GDP on defence today, rather than being forced into a bankrupting amount later.
This is not a call to throw billions at uncertain science and a misapplied Precautionary Principle. Understanding risk is about accepting the limits of our perspectives, while grappling with the odds of the worst case scenario happening, and the costs as well as the strategic consequences of both action and inaction. In some cases, that means accepting that spending might have to go on assets that may never be used – indeed, hopefully just so.
That makes risk management a complex policy area, especially if it means changing from running a just-in-time system towards building up a stockpile. More policy crafters, and certainly more Treasury policy drivers, need to grapple with both the logic and the long odds.
Perhaps a SpAd conference might be in order, to think the unthinkable from their very foundations, and to red game such wild cards. Such a conclave can only take place once the current crisis – long-predicted but somehow still a surprise – finally subsides.
Click here to subscribe to our daily briefing – the best pieces from CapX and across the web.
CapX depends on the generosity of its readers. If you value what we do, please consider making a donation.
