The Office of the Legislative Auditor (OLA) made news when it released a new report on charter schools in Minnesota. (The June 2008 report, simply titled “Charter Schools,” is available at www.auditor.leg.state.mn.us/.)

The finding that got pundits and journalists talking is the conclusion that students in charter schools “generally did not perform as well on standardized academic measures as students in Minnesota district schools.”

So where do we go from here?

If you believe that poverty is the fundamental obstacle to educational performance, you might use this as an occasion to dismiss the role of charter schools, call for a new “war on poverty” that also includes yet more increases in funding for district schools.

It is true that we expect schools to do too much, including saving the planet from environmental catastrophes and teaching parents how to be parents. But let’s not let schools off the hook, either. How a school is run can make a difference.

Finding Adequate Comparisons
Do the findings of the report confirm that the charter school idea has failed? Not exactly. The researchers observed that after various demographic factors are accounted for, “the differences in performance were minimal.”

In other words, charter schools performed at least as good as district schools—even without being able to tap local property owners for taxes.

But let’s back up and start with the authors’ statement that “the limitations of the data do not allow us to make definitive conclusions” about the performance of charter schools or district schools, as a class.

The most important limitation to the report is that it does not have any way to measure what really counts in education: Is a student who starts out the year in a charter school better off at the end of the year or not? If that student is better off than a similar student in a district school, perhaps the charter school idea—or at least that particular school—is good.

In other words, the best way of comparing the value of two schools is to compare how much similar students gain over time. The OLA report did not do this.

This omission is an important limitation to the report. That’s because it is possible that the differences in students who transfer to charter schools may be more important than the differences in the schools themselves.

The OLA attempted to get around this possibility by matching charter and district schools (on student demographics) within their region of the state. Still, it could not find a suitable match for one quarter of the charter schools.

(In addition, nine charter schools with “unique learning programs” such as online curriculums were not included in the comparison. There is a good methodological reason to omit such schools, but the fact that they have unique programs is in itself a benefit to their students.)

Digging deeper into the data reveals something interesting: the relative performance of the two types of schools, at least as revealed in this report, depends on the income and racial composition of the school. In some situations, charter schools as a group do outperform district schools.

For students who were at the very bottom of the economic ladder, district schools held an advantage over charter schools. But for students just below the poverty level, charter schools held their own.

For schools that have overwhelmingly minority enrollments (75 to 100 percent), charter schools held their own.

In addition, charter schools are better than district schools in Minneapolis and Saint Paul in making Adequate Yearly Progress, a measurement of the federal No Child Left Behind education program. That’s important, since these districts are the largest in the state, and among the worst performing. According to Education Week, for example, the graduation rate for 2004-05 was 59 percent for Saint Paul, and 45 percent for Minneapolis. (See the graduation rate map tool, a great new resource, at www.edweek.org/apps/maps/.)

A Lesson from Chicago
So can charter schools work? Other reports suggest the answer is yes. Earlier this year the Rand Corporation (www.rand.org) compared district schools and charter schools in the Chicago Public Schools system. Unlike the OLA researchers, the Rand researchers looked at student gains over time.

They concluded that charter schools “may produce positive effects on ACT scores, the probability of graduating, and the probability of enrolling in college.” These results, they said, “are solidly evident only in the charter [high schools] that also included middle school grades.” In that circumstance, the likelihood of graduating increased by 10 percent (or 7 percentage points); the likelihood of attending college increased by 29 percent.

What made the difference? The Rand researchers said that it was either the charter status of the schools, or their unusual grade configuration—which was in turn made because they were charter schools.

Are charter schools the only cure to what ails public education? Not at all. But lawmakers would be foolish to cap their numbers, as was discussed during the last session in Saint Paul, or make them more like district schools.