More non-rigorous thinking about the arts

This study proves far less than it claims to:

In a report to be released on Monday the nonprofit Center for Arts Education found that New York City high schools with the highest graduation rates also offered students the most access to arts education. The report, which analyzed data collected by the city’s Education Department from more than 200 schools over two years, reported that schools ranked in the top third by graduation rates offered students the most access to arts education and resources, while schools in the bottom third offered the least access and fewest resources. Among other findings, schools in the top third typically hired 40 percent more certified arts teachers and offered 40 percent more classrooms dedicated to coursework in the arts than bottom-ranked schools. They were also more likely to offer students a chance to participate in or attend arts activities and performances.

Growing up in a household where basic experimental biology was how the mortgage got paid, I learned a little about statistics and the principals of data analysis. One of the most basic was: control for the variables.

In this case, that would require asking what else might be going on that would correlate with access to arts education and the required resources. The answer is pretty obvious. Schools in poor neighborhoods are likely to have fewer resources for everything. And schools in poor neighborhoods are also likely to have lower graduation rates for a whole host of reasons, many external to the school.

So a meaningful study of the impact of arts education on graduation rates would have to compare schools in similar situations (poverty rates, ESL levels, free lunch rates, etc.) and demonstrate statistically that the only thing different between schools with different graduation rates was access to arts education. This report didn’t. In fact, the appendix demonstrates very clearly that graduation rates correlate nicely with a whole host of variables, none of which are controlled for.

End result? A meaningless report. Lesson? Just because we’re advocating for the arts doesn’t mean that our arguments don’t have to meet basic standards.