Inner Education for Inward Educators

Do you ever wonder if your news reports on a school’s test scores give you enough information to be useful?  Jay Mathews at Class Struggle does an amazing job looking at the recent uproar of the LA Times planned publishing of school test score data with teacher’s names attached.  He echoes some problems that I see often in education reporting:

The third paragraph of the story prepares the reader for surprisingly awful results from the Times value-added analysis, compared to the school’s high test score averages as reported by the state each year. “Wilbur’s record was among the worst in Los Angeles for boosting student performance in math and English,” the story said.

But a reader eager to see the details backing up this statement has to go 76 more paragraphs before finding it, in this sentence: “On average, students started the third grade in the 77th percentile in math, but by the end of the fifth grade were in the 67th. In English, they slid from 79th to 76th.”

You see my problem? Not only did I have to go almost to the end of the story to get this information, but the numbers do not seem that bad to me. These are after all percentiles, not actual scores. They are dependent on the how wide the scale of measure from first to 99th percentile is–something we are not told in the story. A ten point percentile drop in math is likely significant, but this is an average for the entire school and leaves Wilbur still considerably above average. I doubt that the three-point percentile drop in English is statistically significant at all.

I see this often, a school test scores are reporting as “sliding.”  But how much is a slide?  To be sure this works in reverse, a school district will proclaim bold success over a .5% gain in test scores, and journalists will report a slide of .5% in large headlines.  But how significant is that?

The problem is one of patience, and similar to my previous posts on this subject, schools are best judged by multiple year averages (as teachers should be) not by the capricious movements that occur on a year-to-year basis.