Justice Flunks Math

http://www.nytimes.com/2013/03/27/opinion/when-judges-cant-do-math-justice-suffers.html

Version 0 of 1.

CAMBRIDGE, England

ITALY’S highest court on Tuesday overturned the acquittal of Amanda Knox, accused of the 2007 murder of Meredith Kercher, a 21-year-old British woman who was Knox’s roommate in Perugia, Italy, at the time.

In 2011, an appeals court invalidated the 2009 murder convictions of Ms. Knox, an exchange student from Seattle, and her former boyfriend, Raffaele Sollecito, and released them from jail. Now, Italy’s Court of Cassation has annulled that decision — sending a strange new ripple into a case that has riveted many around the globe for years.

There is enough sensational detail in the Knox case, of course, to keep the tabloid pages filled. At least 10 books have been written about it, including one by Ms. Knox herself, whose memoir is to be published next month. But one aspect of this case — as with so many others, sadly — deserves far more attention than it gets: much unnecessary drama has resulted from bad math.

Miscalculation by judges and lawyers of probabilities, from the odds of DNA matches to the chance of accidental death, have sent innocent people to jail, and, perhaps, let murderers walk free.

The Court of Cassation has not yet publicly explained the motivations behind its ruling. But the appellate judge’s failure to understand probability may well play a role.

One of the major pieces of evidence was a knife collected from Mr. Sollecito’s apartment, which according to a forensic scientist contained a tiny trace of DNA from the victim. Even though the identification of the DNA sample with Ms. Kercher seemed clear, there was too little genetic material to obtain a fully reliable result — at least back in 2007.

By the time Ms. Knox’s appeal was decided in 2011, however, techniques had advanced sufficiently to make a retest of the knife possible, and the prosecution asked the judge to have one done. But he refused. His reasoning? If the scientific community recognizes that a test on so small a sample cannot establish identity beyond a reasonable doubt, he explained, then neither could a second test on an even smaller sample.

Whatever concerns the judge might have had regarding the reliability of DNA tests, he demonstrated a clear mathematical fallacy: assuming that repeating the test could tell us nothing about the reliability of the original results. In fact, doing a test twice and obtaining the same result <em>would </em>tell us something about the likely accuracy of the first result. Getting the same result after a third test would give yet more credence to the original finding.

Imagine, for example, that you toss a coin and it lands on heads 8 or 9 times out of 10. You might suspect that the coin is biased. Now, suppose you then toss it another 10 times and again get 8 or 9 heads. Wouldn’t that add a lot to your conviction that something’s wrong with the coin? It should.

The judge’s rejection of the retest — at least based on the notion that a confirming retest could tell us nothing about the likelihood that the DNA was a match — was a serious error, one that scuppered an opportunity to get at the truth of Ms. Kercher’s murder.

We’ll leave it to others to decide whether Ms. Knox is guilty or not. But the damaging effects of bad judicial math have been clear in other cases.

Flawed testimony by an expert witness helped convict Lucia de Berk, a Dutch nurse accused of murdering several patients, and sent her to prison for six years before her conviction was overturned in 2010.

Prosecutors in the Netherlands had accused Ms. de Berk of killing the sick children and elderly patients, all of whom had first been judged to have died of natural causes, based on the sheer number of fatalities that occurred on her watch. A statistician for the prosecution, using a ludicrous methodology, testified that the probability that the deaths were natural was 1 in 342 million.

It took an exhausting legal fight, two failed appeals and a committee of statistical experts to convince judges that the calculation was deeply flawed — and that Ms. de Berk’s only crime was bad luck.

In a case that shook Britain, Sally Clark, a young lawyer who had lost two babies to crib death (sudden death of an infant without any apparent medical cause) was wrongly convicted in 1999 of having murdered them. Lacking any evidence of abuse, the conviction was based on a maverick calculation by a medical expert who concluded that the odds of two crib deaths’ happening in a family of Ms. Clark’s social status were just one in 73 million. (In fact, double crib deaths occur in Britain every couple of years.)

Ms. Clark’s conviction, as with Ms. de Berk’s, was eventually overturned, but only after the Royal Statistical Society offered a scathing analysis of the calculations made by the prosecution’s witness. Sadly for Ms. Clark, she could not put her life back together. She died of acute alcohol poisoning in 2007, four years after her release from prison.

Decades ago, the Harvard law professor Laurence H. Tribe wrote a stinging denunciation of the use of mathematics at trial, saying that the “overbearing impressiveness” of numbers tends to “dwarf” other evidence. But we neither can nor should throw math out of the courtroom. Advances in forensics, which rely on data analysis for everything from gunpowder to DNA, mean that quantitative methods will play an ever more important role in judicial deliberations.

The challenge is to make sure that the math behind the legal reasoning is fundamentally sound. Good math can help reveal the truth. But in inexperienced hands, math can become a weapon that impedes justice and destroys innocent lives.

<NYT_AUTHOR_ID> <p>Leila Schneps, a mathematician and mystery writer, and her daughter Coralie Colmez are the authors of “Math on Trial: How Numbers Get Used and Abused in the Courtroom.”