The Datum’s the Thing

In his recent blog post, Thomas Martellone takes a methodical walk through the debate around data in education, and provides an excellent primer along with a rational argument. Basically, the concept is that data are useful, but get a bad rap because they are touted as the end all and be all. I agree. The singular of the word, datum, serves as a useful symbol for the fact that data are just a glimpse, from a single angle. The full perspective cannot be gained except by examining many facets and in differing light.

We’ve all learned the Scientific Method at some point, and Martellone gives examples of how to use data safely by investigating conclusions in a similar process: form an hypothesis, test the hypothesis, draw a conclusion. He gives the example of an hypothesis that could have explained an anomaly in some students’ grades, but when he asked the teachers about it, they explained that their process accounted for the divergence. As Martellone highlights, the danger of data in education is when they are taken to be the end all and be all. In fact, they should be part of the story. Data can be useful to inform or facilitate conversation, and used for deeper investigation.

Another current practice that makes “data” a dirty word in education is that they are often misappropriated. Politicians and others call on limited data to draw sweeping conclusions. Martellone reminds us that data should be part of meaningful discussion between committed professionals.

Instead, the single stream of test data– which, taken alone, is not enough to draw thorough conclusions even about student learning, their intended target– is also called upon to shed light on teachers’ performance. Those data are not meant to reflect teacher success or instructional effectiveness. Assessments can be designed for that purpose, but these were intended for evaluating student achievement.

In evaluation, the most informative process is conversation. In the traditional model of administrator observation, I learn more from speaking with teachers about their plans, their process, and the elements of a lesson that did not go well, than I ever will glean from a glance in their gradebook. I want to know how the teacher addresses material that wasn’t learned, whether it’s an instructional failure they intend to fix, or whether it’s a tool their adding to their repertoire to deal with future students’ individual learning challenges. These discussions are also how I impart my values and vision for the school, and the faculty culture, to the newest teachers. Without sharing the stories, we’ll never know anything about each other.

Data are not the enemy; improper or ineffective use of them is. In a conference once, I was told that “Data without analysis is not information. It’s just data.” So true. We must be careful to evaluate and interpret our data carefully, and to always challenge our analyses through dialogue.