Education Tue Oct 04 2011

Finding More in The Consortium's Findings

by Daniel Hertz

To read the headlines, the report released this week by the University of Chicago's University of Chicago's Consortium of School Research says nothing especially surprising: our public school system has shown "No Real Progress" (the Sun-Times) or "Little Progress" (Chicago News Cooperative), and has "Failed Many Students" (the Tribune). (Actually, you might have been surprised if you had been paying close enough attention to know that state tests have been showing big gains in elementary schools, and if you weren't cynical enough to already believe those same tests were being dumbed-down to produce the false appearance of progress. But I assume not very many people fall in that category.)

The actual report, however, contains major news-both good and bad-that either wasn't reported at all by the major outlets, or was relegated to a few lines buried deep inside the story. Here are three points that were under-covered:

1. CPS high schools are much better than they were 10-15 years ago.

Chicago public high schools raised their ACT scores by a full point, from an average of 16.1 in 2001 to 17.1 in 2009. This is at the same time as the number of students taking the test-including black, Latino, and low-income students-rose dramatically. In fact, taking into account the changing demographics of test-takers, the researchers found the growth was about 10% faster than it appears from the nominal scores. Not only that, but the improvement remains even if you hold the levels of achievement of entering 9th graders constant-meaning we're not just seeing higher-performing seniors because high schools are matriculating higher-performing freshmen. High schools are doing a better job of teaching.

Graduation rates also rose sharply, from 48% in the mid-1990s to 66% in 2009. The researchers investigated whether this might be the result of fudged numbers, but found the results held up to scrutiny: again, this was a real gain. In fact, CPS pushed up its graduation rate even as it held its graduates to higher course standards. The number of graduating high schoolers taking college-prep course loads rose sharply, especially among black and Latino students-fewer than 60% of whom took three years of math in 1997, compared with over 90% today. Meanwhile, the number of CPS graduates passing an AP class nearly tripled.

The flip side of this good news is that in most cases, there was more progress for white and Asian students than their black or Latino counterparts. On the ACT, for example, black and Latino students grew by an average of 0.9 points over the last decade, while white and Asian students made just over a full point of gains. In other words, the achievement gap, already large, got a little bit bigger. The report suggests that part of this might be due to the "Renaissance 2010" policy started under Arne Duncan, which led to huge number of school closings, mostly on the South and West sides at heavily black schools. Previous research suggests those disruptions hindered learning for affected students.

Still, though no one should be taking a victory lap any time soon, these are impressive results. And it is frustrating that they're not being recognized by major Chicago media outlets. In my experience, both students and teachers in underperforming schools bristle under the constant drumming of the narrative of failing public education-that it has always failed, always will fail, and there is nothing any of us can do about it. No school, and certainly none of the young human beings in it, is hopeless. To the extent that the headlines following this report perpetuated the narrative of failure by ignoring the very real progress that CPS high schools have made, it's insulting to the students, teachers and administrators who deserve better.

2. The report has concrete evidence that underperforming schools are using test tricks, rather than actual teaching, to game state tests-and it's working.

This won't shock anyone who's been involved with low-performing public schools in the last ten years, but the clarity with which UCCSR demonstrates, and measures, this phenomenon is worth some attention.

What exactly is the evidence? Well, in 2006, Illinois switched its state test from the Iowa Test of Basic Skills (ITBS) to the Illinois Standards Achievement Test (ISAT), which have somewhat different formats and types of questions. Relatively high-performing schools took the change in stride; their students continued making about a year's worth of academic growth, as they normally do.

But students at low-performing schools, where high-stakes tests generate weeks, if not months, of intense test prep, did something very strange. They averaged zero years of growth between the last ITBS and the first ISAT, despite the fact that every year before and after 2006 they had also been making roughly a year of progress. It was as if they had learned nothing at all.

The authors of the study concluded that "scores dipped in 2006 because schools had developed instructional techniques that were specifically targeted to the ITBS, and these techniques did not carry over to success on the ISAT." In other words, a child who scores at a 3rd-grade level at a low-performing school might actually, as measured by another test, only have 2nd-grade level skills: test prep, as opposed to actual curricular instruction, adds a full grade that turns out to be worthless if you're not taking that particular test.

This raises-not for the first time-serious questions for supporters of accountability-based policies. The school reform movement's theory of progress depends on high-stakes testing to measure learning and hold underperforming teachers and administrators accountable. But the very act of "holding people accountable" to test scores turns out to decrease learning-or, at the very least, it leads to the kind of manipulation that makes test scores unreliable. Again, none of these criticisms are new. But news organizations ought to be using this report to force Chicago's pro-reform public officials to reckon with these problems honestly.

3. Illinois state standards are obscenely low.

The number of questions required to "meet expectations" on the ISAT has decreased from 36 in 2006 to 32 in 2010. Illinois officials claim this is because the questions have gotten harder; the researchers, having analyzed the tests, do not agree.

In any case, most students who "meet expectations" are graduating completely unprepared for college. The report found that only about 60% of students who score at the top of the "meets expectations" range-that is, close to the 90th percentile of testers statewide-get a 20 or higher on the ACT, the lowest required score to have a good chance of admission at state colleges. Students at the bottom of the "meets expectations" range, in the words of the report, "have nearly no chance" of admission.

This is another challenge for anyone who wants to use tests to measure academic proficiency, although there's a much simpler solution: raise standards. If Illinois wants to be graduating students who are college-ready, don't tell parents-and everyone else-that students who "have nearly no chance" of admission to college are doing fine.

In short, anyone who wants to believe that there are no signs of progress in Chicago's public schools needs to adjust their cynicism in light of these numbers. Many more high schoolers are graduating, taking harder classes, and getting better ACT scores than ten years ago. At the same time, the testing/accountability regime that has been in place at CPS for well over a decade needs to come up with an answer for yet another piece of evidence that it creates perverse incentives for teachers, administrators and students. But first, this news needs to make it to the city's major media outlets.

Daniel Hertz recently returned to his native Chicago after a six-year absence that took him to Montreal, Boston, Mexico, Russia, and Memphis, among other places. He is glad to be back.