This article is from the source 'guardian' and was first published or seen on . It last changed over 40 days ago and won't be checked again for changes.

You can find the current article at its original source at http://www.guardian.co.uk/politics/2012/jun/20/test-policies-randomised-controlled-trials

The article has changed 2 times. There is an RSS feed of changes available.

Version 0 Version 1
Test policies in 'randomised controlled trials', nudge unit urges Test policies in 'randomised controlled trials', nudge unit urges
(40 minutes later)
The government should be more rigorous in assessing the impact of its policies to make sure they work and are cost-effective, a new Cabinet Office report by the Behavioural Insights Team, or "nudge unit", has advised. The government should be more rigorous in assessing the impact of its policies to make sure they work and are cost-effective, a new Cabinet Office report by the behavioural insights team, or "nudge unit", has advised.
Better evidence on whether new policies work, gathered through more robust trials, could save public money and stop inadvertently harmful policies, the report entitled Test, Learn, Adapt cautioned. It comes as the government has come under fire from experts for ignoring evidence on some of its flagship policies on education and welfare reform.Better evidence on whether new policies work, gathered through more robust trials, could save public money and stop inadvertently harmful policies, the report entitled Test, Learn, Adapt cautioned. It comes as the government has come under fire from experts for ignoring evidence on some of its flagship policies on education and welfare reform.
New drugs and medical treatments are required to pass a randomised controlled trial (RCT), showing they are better or more cost-effective than any existing medicine, before they are prescribed by the NHS. But policies, such as on unemployment schemes, education, tax reminders or parenting programmes, are not subject to the same checks and thus there is often no evidence as to whether they actually work.New drugs and medical treatments are required to pass a randomised controlled trial (RCT), showing they are better or more cost-effective than any existing medicine, before they are prescribed by the NHS. But policies, such as on unemployment schemes, education, tax reminders or parenting programmes, are not subject to the same checks and thus there is often no evidence as to whether they actually work.
The report said these random trials could be applied to "almost all aspects of public policy".The report said these random trials could be applied to "almost all aspects of public policy".
An RCT is a test in which individuals who might be subject to a new programme – for example a "back to work" training scheme – are selected at random to receive the proposed intervention. They are then compared with another group of individuals who are treated as normal (a "control" group). The two are then compared to see which scheme worked best.An RCT is a test in which individuals who might be subject to a new programme – for example a "back to work" training scheme – are selected at random to receive the proposed intervention. They are then compared with another group of individuals who are treated as normal (a "control" group). The two are then compared to see which scheme worked best.
The use of these trials has rocketed in medicine, up from fewer than 20,000 in the 1960s to more than 200,000 in the last decade, but for other programmes the increase was much smaller. The report recommended starting the trials in uncontroversial areas – such as wording on tax reminders (one trial suggested different wording could improve effectiveness by around 30%) or frequency of visits to jobcentres – before working up to more contentious issues.The use of these trials has rocketed in medicine, up from fewer than 20,000 in the 1960s to more than 200,000 in the last decade, but for other programmes the increase was much smaller. The report recommended starting the trials in uncontroversial areas – such as wording on tax reminders (one trial suggested different wording could improve effectiveness by around 30%) or frequency of visits to jobcentres – before working up to more contentious issues.
Dr Ben Goldacre, a science blogger and research fellow at the London School of Hygiene and Tropical Medicine who is one of the study's external co-authors, said as the government was required to evaluate its policies, it might as well do it well.Dr Ben Goldacre, a science blogger and research fellow at the London School of Hygiene and Tropical Medicine who is one of the study's external co-authors, said as the government was required to evaluate its policies, it might as well do it well.
"Almost all new policies get assessed, but currently they're assessed rather badly," he said. "We should do it right. In the 1970s, there were lots of doctors and surgeons around who aggressively opposed random trials on drugs, as they knew what was best for their patients.""Almost all new policies get assessed, but currently they're assessed rather badly," he said. "We should do it right. In the 1970s, there were lots of doctors and surgeons around who aggressively opposed random trials on drugs, as they knew what was best for their patients."
Goldacre warned that politicians too often picked policies for reasons of ideology rather than whether they would prove effective or not.Goldacre warned that politicians too often picked policies for reasons of ideology rather than whether they would prove effective or not.
"An important first step is humility," he said. "To recognise you don't know if an intervention will be effective. There's a lack of humility in policy-making. People often have to state things in very melodramatic, black and white terms. "An important first step is humility," he said. "To recognise you don't know if an intervention will be effective. There's a lack of humility in policy-making. People often have to state things in very melodramatic, black-and-white terms.
"RCTs let you check whether your policy is having the outcome you want. This allows you to identify policies which don't work – and if things don't work, we should probably stop them. It's not just about saving money, it's about reducing harm.""RCTs let you check whether your policy is having the outcome you want. This allows you to identify policies which don't work – and if things don't work, we should probably stop them. It's not just about saving money, it's about reducing harm."
The government has come under fire in recent weeks for expanding some of its policies despite an absence of evidence to suggest they work. The Department for Work and Pensions expanded a scheme requiring selected unemployed people to do up to 30 hours unpaid work a week for up to four weeks from a target of 10,000 people a year to 70,000 – despite its own study concluding the programme had "no impact on the likelihood of being employed".The government has come under fire in recent weeks for expanding some of its policies despite an absence of evidence to suggest they work. The Department for Work and Pensions expanded a scheme requiring selected unemployed people to do up to 30 hours unpaid work a week for up to four weeks from a target of 10,000 people a year to 70,000 – despite its own study concluding the programme had "no impact on the likelihood of being employed".
Similarly, sweeping reforms to the UK's national curriculum introduced last week by Michael Gove were attacked by one of his expert advisers for introducing "overly prescriptive" changes, while under the previous government drugs adviser Professor David Nutt was dismissed after he made blunt public remarks that the UK's drugs rating system had no basis in evidence.Similarly, sweeping reforms to the UK's national curriculum introduced last week by Michael Gove were attacked by one of his expert advisers for introducing "overly prescriptive" changes, while under the previous government drugs adviser Professor David Nutt was dismissed after he made blunt public remarks that the UK's drugs rating system had no basis in evidence.
The nudge unit's report suggested that the financial crisis might help create a good climate for introducing randomised trials – if interventions were already being phased in small areas at a time due to resource shortages, this provided a good opportunity for "natural" experiments to better see which worked.The nudge unit's report suggested that the financial crisis might help create a good climate for introducing randomised trials – if interventions were already being phased in small areas at a time due to resource shortages, this provided a good opportunity for "natural" experiments to better see which worked.