Documents show that system ignored flawed information in cheating investigation

The Baltimore Sun:  Experts critical of methods of investigation

According to documents obtained by The Baltimore Sun this week, independent hearing officers — attorneys hired by the system to render facts and findings in personnel hearings — found the school system's investigation, triggered by plunging student test scores, to be fundamentally flawed.
"While there is a healthy dose of speculation, suggestion and suspicion that cheating occurred in 2009, there is a lack of credible evidence that cheating actually occurred," one hearing officer wrote. "Not one witness was able to testify with certainty that they knew what happened in Abbottston. To the contrary, witness after witness stated that they did not know what happened at Abbottston."
The hearing officers also said city schools CEO Andrés Alonso knew there were problems with a state analysis of eraser markings in test books but moved ahead anyway to dismiss the principals last year.
The findings also raise questions about how the system determines whether a school cheated and who is responsible...
...The most troubling aspect of the school system investigation, according to the hearing officers, was that the system built its case on a faulty erasure analysis. In his announcement that Abbottston had cheated last year, Alonso said the analysis showed patterns of erasures from wrong to right answers "beyond the realm of probability."
In their findings, the hearing officers called the erasure analysis "incompetent" and "crude," and outside experts have called it an embarrassment to the scientific profession. The erasure analysis, conducted by a longtime state Education Department official, was done manually in a living room. Any documentation that could have authenticated it was destroyed....
...Robert Wilson, the lead investigator of a cheating scandal in Atlanta, considered the largest in the history of U.S. public education, said the Abbottston scores should never have been invalidated based on the analysis because it was so flawed. He reviewed hearing documents in the Baltimore case.
Wilson led the 2011 investigation that uncovered rampant cheating in Atlanta public schools — where his team issued a 171-page report identifying 150 educators, 82 of whom confessed to cheating.
Atlanta's erasure analysis consisted of reviewing more than 100,000 test booklets to establish a benchmark, electronically scanning the test booklets and a manual analysis to verify the results.
"If you showed me 167 booklets, and I did exactly what she did, I might be able to say, 'This is a hell of a lot of erasures,' but I'd have to have a lot more than that," Wilson said.
"I am surprised that someone who claims to a psychometrician would have been willing to draw conclusions from such unreliable work," he added. "That the Baltimore City Public Schools would have relied upon it is even more surprising."...