The Guardian view on statistics in politics: over-counting
|
Version 0 of 1. In Britain “empiricist” is a compliment, a fancy way of saying a good egg with an open mind. The universities shun the grand theories and lofty principles of the continent in favour of “the evidence”. The citizenry expects leaders to focus on “outcomes” rather than means. “Ideological” has become a term of abuse in the political trade. Start out with this culture, and pour on the big data churned out by ubiquitous, interconnected computers, and – past generations might have imagined – you’d end up with a form of public life as problem solving. Things would be tried, results measured and conclusions teased out in seminar-style discussions. But in reality, while the 2015 election will certainly be the most fact-blitzed in history, fact-led pragmatism has not arrived. Instead, the coming campaign will be yah-boo with numerical knobs on. Related: Celebrate statistics as a vital part of democracy | Letters Labour will trumpet dire statistics about real wages, which rest on the discredited old retail price index. The Conservatives will come up with dodgy definitions of family incomes, which include those taxes that they have cut, but exclude those they have increased, and ignore benefit cuts. Nobody else will be left any the wiser. Swing voters will not be swung, but instead sigh that there are lies, damned lies and statistics. Mean, median and perhaps even modal averages will be arbitrarily seized on to support every self-serving point, about everything from A&E waits to the proportion of crimes that the police are clearing up. From high on the mountain and above the fray, Sir Andrew Dilnot of the UK Statistics Authority, or the Institute for Fiscal Studies, will be called to pass Solomonic judgments on the spin. It won’t make much difference. By the time the referees’ rulings are in, the circus will have moved on. Or else the politicians will simply brazen it out: only this week David Cameron has been bandying about statistics to imply the very causal connection between the government’s benefit cap and claimants moving into work that Sir Andrew had last month cautioned the Department for Work and Pensions against assuming. The abundance of data does nothing to dull the motive to abuse numbers, and may actually increase the opportunity. But there are also deeper problems with figure-fuelled discourses, even when there is no intention to deceive, as Thursday’s controversy about school league tables showed. Since being introduced in 1992, there have been so many reinventions that it is no longer easy, well, to keep count. The first generation were crude exam pass rates, which told everything about who went to a school and little about how good a job it did. Next came so-called value added tables, which started with pupils’ attainment on the way in, and then tried to assess how far they’d been stretched by the time they left. That sounded sensible but the baseline proved to be wobbly, and schools with challenging intakes protested that the negative peer effects produced by poverty and other disadvantaging demographics were still not being fairly reflected. This led to the birth of “contextual value added” tables that adjusted for these things. Problem solved! Except, of course, that the tables were no longer intelligible. They sweated the data to assess how institutions fared after allowing for every circumstance, which may be interesting for some but not for parents, who are keen to know how good schools actually are – without allowing for anything. All this was under Labour: Michael Gove turned the tables again. He stripped out some of the baffling context and made other tweaks, such as discounting the once-vaunted iGCSE in English, which was what led to Thursday’s farce of Eton and Westminster being shunted to the bottom rank. In every flavour of table yet tried, many of the bumps up and down the league each year can be explained by random variation in the pupil mix. That is to say that the noise overpowered the signal, just one of the perennial perils in the statistical trade. Others include the sort of malign warping of definitions seen in George Osborne’s “tax transparency” letters, and confusion about what question it is that a figure is actually supposed to settle. The project of replacing a clash of ideas with a policy calculus was always dubious. Anyone still hankering for it should admit their number’s up. |