17 February 2021

Government must get back to (data) basics


Database management, like productivity, is one of those topics that automatically hits the mental snooze button. It may be hugely important, but it’s also almost guaranteed to make the eyes glaze over.

Which is why I want to get your attention by saying something provocative: the central lesson of the pandemic is that database management is both the most important part of modern government, and its most intractable limitation.

Throughout the pandemic, to a rough approximation, all of the UK’s biggest policy successes have been built on good databases. And where things have gone wrong – as they so often have – it is invariably a bad or nonexistent database that is to blame.

The furlough scheme, for example, worked because the Treasury could simply (or not so simply) reverse-engineer PAYE, sending the money flowing in the opposite direction. The expansion of Universal Credit to cover the newly unemployed depended on the smooth functioning of the Universal Credit database – and, as with other examples, on the fact that cloud computing allowed it to be expanded and accessed without additional stress.

The shielding process was, essentially, a database management exercise which involved crunching together six separate national datasets. Then, of course, there is the triumph of the vaccine rollout – built on the robust patient records held by GPs and the NHS, which enabled them to separate the population by age and vulnerability.

Even Kate Bingham and her team, when deciding which vaccines to order, started by compiling a database of all the candidates in development – and then filtering by which were the likeliest to come on stream in 2020.

Now think about where things have gone wrong. Covid test results being lost because the data had been cobbled together using Excel? Track and Trace, where you were essentially trying to build a database mapping the infected’s social networks from scratch? The initial realisation that the Government couldn’t actually message the entire population via their mobiles, and had to beg the mobile operators to act on its behalf? All ultimately involved databases, or rather the lack of them.

The lesson of the crisis, in other words, is very clear. A policy is easy to do if you can adjust an existing database easily, and almost impossible to do if you can’t. But that’s a lesson that pretty much no one outside Westminster realises.

For example, when Labour called for key workers such as teachers to be prioritised for the jab, the rebuttal – correctly – was that it would result in more older people dying. (Since the evidence is that risks to teachers are substantially lower than for the existing vulnerable groups.) But an equally good rebuttal could have been that there was no database of ‘teachers’, let alone ‘key workers’, that would be suitable for the task – let alone one which could be plugged into the NHS systems.

To give another example, the debate over the £20 uplift in Universal Credit has proceeded as if the policy comes first. In fact, as the Treasury and DWP will tell you, all ideas for what could or should be done as an alternative first had to pass the test of being compatible with the existing computer system.

Universal Credit itself, in fact, is a great example of this new world. It is normally thought of as a policy idea – to consolidate multiple existing benefits in one. In fact, it was an exercise in database management: attempting to plug the benefits and the tax databases into one another so that the two could be seamlessly linked. One of the main reasons it took years and billions of pounds more than expected was that this was rather more difficult than first envisaged, particularly in terms of dealing with all the complexities and wrinkles and edge cases of human experience that defy easy categorisation.

If you accept this thesis, that database management has become either a core task of government or perhaps even the core task, it has some pretty important consequences.

The first is that if you are making policy, the very first question you have to ask is ‘What existing database can I use, and will it do what I want it to?’

If there isn’t a clear answer, then that policy either won’t happen, shouldn’t happen, or will be far more expensive to make happen than you realise.

For example, I’m seeing a load of people talking about vaccine passports as a civil liberties issue, and pretty much no one asking ‘What database would they use?’ – which is utterly essential to them actually happening, if they ever do.

But there’s also a huge issue around the state and number of databases we have – after all, one of the oldest maxims in computer science is garbage in, garbage out.

I was one of those in the trenches with No2ID back in the day. I’m still instinctively hostile to ID cards. But one of the biggest problems the British state has it that it has an insane number of databases that do not really talk to each other. Verify, Govt Gateway, HMRC/NI numbers, DWP, NHS numbers (plus separate for Scotland, Wales, NI), passports, driver’s licences, learner and pupil numbers in schools, blood donors, criminal records, Disclosure and Barring, the electoral roll… the list goes on and on.

If you talk to Estonians, the reason that they are streets ahead on digital government is that everyone has a unique and mostly comprehensive digital identity, which plugs into everything else. Talking to ministers about this pre-pandemic, it was top of the digital government wishlist.

But of course, we already have some pretty good examples of how our government can screw up attempts to mash databases together like this. Verify was itself an attempt to unify much of this data and create a single digital identity for citizens. It hasn’t really been going well, to the point that HMRC essentially declared UDI. Then there was Universal Credit, the NHS IT system, etc.

Another issue is the risk that we end up confusing the map and the territory. A big reason that HMT/HMRC haven’t done as much to help the self-employed in this crisis (and are generally hostile to them) is that they aren’t on the databases in the same neat way as those earning via PAYE.

There’s also the question of digital exclusion – if everything is done via database, you’re going to push citizens towards digital interactions which are nice and easy to store (and obviously a lot cheaper)

To see the potential consequences, read this fascinating article from Tom Forth on bin apps, which shows how local government has been bypassed on this stuff in favour of national – and how he has accidentally ended up with the power to send information, or misinformation, to almost 100,000 people.

The future of policy, in other words, is not just about ideas, but databases. Good ones, bad ones, new ones, old ones. Even at this point, they are utterly crucial to public service delivery – and will only become more so, given that pretty much all the benefits of artificial intelligence depend as much on the quality of the databases the algorithms are given to ingest, as the quality of the algorithms themselves.

And one of the central challenges we face is that fixing Britain’s databases could be transformative for public services – or a complete and utter car crash.

Click here to subscribe to our daily briefing – the best pieces from CapX and across the web.

CapX depends on the generosity of its readers. If you value what we do, please consider making a donation.


Recurring Payment

Thanks for your support

Something went wrong

An error occured, but no error message was recieved.

Please try again, or if problems persist, contact us with the above error message. We apologise for the inconvenience.

Robert Colvile is Director of the Centre for Policy Studies, a Sunday Times columnist and Editor-in-Chief of CapX.