January/February 2008

Leadership Lessons From Schools Becoming "Data Wise"

by Jennifer L. Steele and Kathryn Parker Boudett

When delivering her opening-day speech to faculty at McKay K–8 School in Boston, second-year principal Almi Abeyta hoped that displaying recent state test results would “light a fire” among teachers and spark a powerful conversation about instructional improvement. Instead, teachers reacted with stunned silence, quickly followed by expressions of anger and frustration. It was the first they had heard about the prior year’s decline in language arts scores. Almi felt as if she “had dropped a bomb” on the room. Far from igniting collaborative energy, her presentation of achievement data seemed to have squelched it.

As schools respond to external pressure to raise student achievement, the perils of examining data loom large. How, school leaders may wonder, do you convince colleagues that engaging in ongoing, collaborative data discussions is worthwhile? How do you discuss data and instruction without finger-pointing or leaping to conclusions? And how do you use insights gleaned from the data to make meaningful—and lasting—instructional improvements?

A few years ago, we collaborated with a team of professors, school administrators, and graduate students to write Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning (Harvard Education Press, 2005). The book offers an eight-step approach to collaborative, evidence-based instructional improvement (see “The ‘Data Wise’ Improvement Process,” HEL, January/February 2006). Since then, schools all over the country have adopted the Data Wise approach. As we worked with many of them, we realized that teachers and administrators who are spearheading the Data Wise improvement process in their schools—as well as those pursuing other approaches—often encounter similar questions and obstacles. So we set out to develop case studies of eight of these schools, documenting the leadership challenges that school leaders typically face during each step of the improvement process, as well as the strategies they use to address them.

Investing in Preparation

In the first phase of the Data Wise process, Prepare (see “The Data Wise Improvement Process”), school leaders typically face two critical challenges: communicating the need for a data initiative and creating data teams that are equipped to lead the work. The leaders we studied confront these challenges in two ways: by making data relevant, and by giving their data teams time to develop the skills and systems they need to be successful.

Make data relevant. As school leaders embark on the improvement process, they need to convince staff that looking at data will not be yet another distraction from their work but will help them do that work more efficiently. For instance, when taking the helm of Newton North High School in Newton, Mass., a school with a history of high academic achievement, first-time principal Jennifer Price found herself in a situation where test scores could easily be dismissed as beside the point. She decided to focus on a topic of longstanding concern to both faculty and the community: how to close the school’s academic achievement gaps. This helped her recruit a large, diverse team of faculty members to gather and analyze data. Explaining her decision to make data relevant, Jen says, “Every department sees the achievement gap manifested in one way or another. By focusing the work of the data team on the achievement gap, the use of data becomes connected to why people come to work.”

Set aside time to build capacity. In addition to establishing data teams, school leaders need to give team members time to develop their knowledge and to create systems that support the team’s efforts (see “Is Your School Ready to Become ‘Data Wise?’”). Shortly before undertaking the Data Wise improvement process, Pond Cove Elementary School in Cape Elizabeth, Maine, had emerged from a cumbersome, externally imposed assessment initiative that was ultimately suspended. Principal Tom Eismeier knew that if the Data Wise approach was to be successful, he and his data team would have to think carefully about how to get the process right. As media specialist Shari Robinson recalls, “[We] didn’t want it to end up as just another failed initiative.” Consequently, Tom, Shari, and the rest of the data team spent a semester in preparation. They took inventory of data already in use at the school, developed a computer-based data analysis system that would be easy for teachers to use, and chose an instructional focus—literacy—that the faculty had already made a priority for the year. Although the team often felt they were losing a race against the clock as time wore on and the most recent test data grew stale, their patience paid off in the end, when their user-friendly approach to data analysis was well received by their colleagues.

Facilitating Large-Scale Inquiry

In moving from the Prepare to the Inquire phase, school leaders often face another critical challenge: how to engage the entire faculty in honest conversations about data, particularly when, as Shari Robinson puts it, “Data can wound.” This was the challenge Almi Abeyta encountered in presenting her data to the McKay School faculty. In addressing that challenge, Almi and other leaders we observed demonstrate two important lessons: establish clear norms for looking at data, and conduct frequent, focused conversations about student learning.

Establish clear norms for data analysis. At McKay, Almi bounced back from her initial presentation and learned to lead productive data conversations by creating a transparent, nonthreatening discussion process. Adapting a protocol commonly used to analyze visual art, she and her data team now present test score data graphically during faculty meetings and ask teachers to ground their data interpretations in objective observations. With its focus on observation and objectivity, this approach facilitates rich conversations and minimizes the threat of finger-pointing or blame.

Conduct frequent, focused conversations about student learning. At Murphy K–8 School in Boston, principal Mary Russo and her staff also rely on clear norms to promote inquiry. They have developed a structured peer-observation protocol in which the teacher who is being observed chooses the lesson, briefs colleagues beforehand, and specifies the aspects of the lesson on which she would like feedback. This protocol puts teachers at ease during the potentially threatening experience of being observed by their colleagues and makes it easier to conduct peer observations on a regular basis. Murphy second-grade teacher Tricia Lampron recalls the first time she participated in this process: “If there were no steps or predesigned process, I wouldn’t have known how to prepare or what my peers would be watching. But the structured process provided an opportunity to focus the observation. . . . That made all the difference.”

Taking Meaningful Action

In moving into the Act phase, Data Wise leaders face the challenge of helping faculty choose, implement, and assess a viable action plan based on insights from the data they have gathered. Taking action can prove difficult; faculty members often have divergent ideas about how broad or narrow the action plan should be and what kinds of instructional improvements are likely to have the most impact. The schools we observed address this challenge by getting down to the “nitty-gritty” in their action planning and by helping teachers “keep the faith” when refinements are needed.

Get down to the “nitty-gritty.” When test scores at Mason Elementary School in Boston showed that students were struggling with writing about texts, teachers were shocked. After all, students wrote about texts all the time in their readers’ notebooks. However, when teachers examined the notebooks collaboratively, they realized that each teacher had different standards for evaluating students’ reading-response letters. As in many schools, a key challenge the teachers faced was defining consistent instructional expectations across grades. After much conversation and debate, they developed an action plan that described exactly how they would teach and assess reading-response letters at each grade level. Teacher and data coordinator Hilary Shea explains that this “nitty-gritty” focus was the key to the plan’s eventual success: “If you want improvement . . . you can’t tackle everything at once. Getting people to choose small topics is so important.”

Keep the faith. The Data Wise improvement process is not a one-time event but a model of ongoing inquiry. The school leaders we observed in our case studies understand that the work of continual improvement is never done. At Community Academy, an alternative high school in Boston, principal Lindsa McIntyre and her faculty devised an action plan for assigning homework consistently across the school. However, in assessing the plan’s implementation and effectiveness, they realized that their initial success in raising teachers’ expectations and students’ engagement was being eroded by the ongoing transfer of new students into the school, with some classes doubling in size. Some new students resisted doing homework, while others found the requirement overwhelming and despaired of keeping up. Lindsa and her team realized they had to explore new alternatives: Establish a study hall? Require new students to start on Mondays, so teachers could plan orientation activities? The challenge for Lindsa and the leadership team—as for any school leader at this phase of the cycle—is to take heart from evidence of success while continuing to target areas for improvement.

Learning from Leaders

The leaders in our eight case studies creatively adapted the Data Wise improvement process to meet the unique challenges facing their schools. At the same time, they drew many of the same lessons from their experiences, based on their common commitment to shared leadership, collaborative learning, and evidence-based decisionmaking.

As for Almi Abeyta, the lessons she learned from her initial presentation fueled her determination to foster productive, collaborative data conversations among the faculty. Two years later, she was able to turn the opening-day presentation over to her enthusiastic data team, who presented evidence of academic improvement in several areas and then announced that McKay had made Adequate Yearly Progress in language arts. On hearing the news, teachers cheered. Then they dove right into a spirited discussion of how to build on their students’ progress in the coming year.

Jennifer L. Steele is a doctoral student at the Harvard Graduate School of Education. Kathryn Parker Boudett teaches at the Harvard Graduate School of Education and is the director of the Data Wise Project.

Data Wise in Action, edited by Kathryn Parker Boudett and Jennifer L. Steele (Cambridge, MA: Harvard Education Press, 2007), $29.95. To order, call 1-888-437-1437 or visit harvardeducationpress.org.