2 August 2022

Sadiq Khan is backing obesity claims based on shameless junk science

By

Has banning junk food adverts on the Tube dramatically reduced obesity and saved the NHS £millions?

That’s the claim made in a study published today – ‘The health, cost and equity impacts of restrictions on the advertisement of high fat, salt and sugar products across the transport for London network: a health economic modelling study’ – which seems destined to go down in history as fact. It found that TfL’s 2019 ban on advertising food deemed to be high in fat, sugar or salt (HFSS) is ‘estimated to have resulted in 94,867 (4.8%) fewer individuals with obesity’ and will ‘save £218 m in NHS and social care costs over the lifetime of the current population’.

In a press release which accompanied the study’s publication, the Mayor of London said:

‘This study, which builds on research from earlier this year, demonstrates yet again that the ground-breaking restrictions we introduced could not only influence behaviour and ultimately save lives but could directly save our NHS hundreds of millions of pounds.’

Under the headline ‘Junk food advertising ban on Transport for London has stopped 100,000 people from becoming fat, study claims’, the Daily Mail published a graph (taken from the study) which seems to show obesity rates in London before and after the policy intervention.

The authors of the study – from Sheffield University and the London School of Hygiene and Tropical Medicine – didn’t do anything as gauche as count the number of obese people to arrive at this finding. No people were studied in the study. Their research, such as it is, is wholly based on theoretical modelling, and the key assumption in the model is a factoid from another modelling study.

The factoid is that London households consumed 1,000 fewer calories from HFSS food after the ban came into effect. This is deeply implausible. It is a much, much greater calorie reduction than the Department of Health expect to be achieved from a nationwide pre-watershed broadcast advertising ban and a total online advertising ban combined.

The study that arrived at this claim is one of the most shameless pieces of junk science I’ve ever seen published in a journal. It used Kantar household consumption data, which doesn’t include restaurant meals, fast food or snacks bought outside the home – i.e. a large proportion of the HFSS food that gets advertised – and compares food sales in London with food sales in the north-east of England.

The north-east is supposed to be the control group, and was chosen because the people there are least likely to suffer from ‘contamination through regular commuting to London’, as the authors put it.

But it turned out that consumption patterns before and after the ban weren’t much different in the north-east than they were in London. Both groups increased the amount of chocolate and confectionery they bought. Both groups reduced the amount of sugary drinks and sugary cereals they bought. The Londoners slightly decreased their purchases of puddings and biscuits while the Northerners slightly increased them. The Northerners slightly reduced their purchases of savoury snacks while the Londoners slightly increased them. 

It was all much of a muchness and there was nothing in the data to suggest that ‘ground-breaking restrictions’ had been implemented in either region.

At this point, the authors essentially abandoned the idea of using the north-east as a control group and instead made up some counterfactuals, imagining how much HFSS food Londoners would have bought if the ad ban hadn’t been introduced. They then compared the recorded consumption figures with the figures from the counterfactuals and, voila!, the difference was attributed to the ban.

Creating a counterfactual is always more of an art than a science. The credibility of the findings depends on how credible you find the counterfactual. In this instance, they were not remotely credible. They were wildly fantastic.

Take the model for chocolate and confectionery, for example, which is an important part of the 1,000 calories that were supposedly dodged after the ban came into effect.

.

The orange line is the north-east. The blue line is London. The dotted blue line is an imaginary London without an ad ban. For reasons that the authors do not come close to explaining, chocolate consumption in London, which had previously been well below that of the north-east, would have suddenly surged to the same level as the north-east in August 2019. By this method, the authors turned a 42 calorie increase in chocolate and confectionery purchases into a 318 calorie reduction.

This is pure fantasy, but it was published in a journal and presented as fact.

Today’s study merely extrapolates from that fantasy. The reduction in obesity and the healthcare savings it supposedly yields only exist in a model. There is little point discussing the econometric and health modelling because it starts from a totally false premise. It’s a classic case of garbage in, garbage out.

We still don’t have the adult obesity statistics for 2020/21, but we do have the statistics for children and they are broken down by region. The Government’s definition of childhood obesity is horribly flawed and includes a large number of children who are not actually fat. Nevertheless, it is based on body mass index (BMI) and it does show us how many children are exceeding a certain weight threshold.

Rates of childhood obesity rose everywhere in 2020/21 thanks to the lockdowns, but they rose most sharply in London. In England as a whole, obesity among 11 year olds rose from 21% to 25.5% but rates shot up twice as fast in London, from 23.7% to 30%.

.

London has now overtaken the West Midlands as the region with the highest rate of childhood obesity.

I am not, of course, suggesting that London saw the biggest rise in child obesity in 2020/21 because it had an HFSS food advertising ban on public transport, but these figures are hardly consistent with a 4.8% decline in overall obesity.

Regardless of what you think of this particular policy, it is worrying that public health policy-making has become so divorced from observable reality. Policies are proposed on the basis of modelling, evaluated on the basis of modelling, and the modelling is carried out by advocates of the policy. At no point are facts allowed to intrude. A rise in chocolate consumption becomes a fall in chocolate consumption. A rise in obesity becomes a fall in obesity. Activist-academics have created a world of pure imagination and are exploiting the broken peer-review process to drag us all into their land of make believe.

Click here to subscribe to our daily briefing – the best pieces from CapX and across the web.

CapX depends on the generosity of its readers. If you value what we do, please consider making a donation.

Christopher Snowdon is the Head of Lifestyle Economics at the Institute of Economic Affairs.

Columns are the author's own opinion and do not necessarily reflect the views of CapX.