January/February 2005

Buy this issue

Randomized Trials in Education

by Beth C. Gamse and Judith D. Singer

Many educators are unsure about what a randomized trial actually entails. In the simplest randomized trial, individual students, teachers, classes, or even whole schools are randomly assigned to one of two (or more) groups. One group will test a new program or intervention. The other will continue to use the same program or intervention they were already using. Because of random assignment, we can assume that the groups are comparable before the new program is implemented; any statistically significant difference in the focal outcome between the groups can therefore be attributed to the new program. In this way, an experiment quantifies the value added of a new program: how much better students do than they would have done under the standard program.

One example of the confusion that surrounds randomized experiments is the experience one of us had when recruiting states to participate in a large trial. A chief state school officer agreed to participate, in part because he wanted his state to be a leader in the research arena. He explained that as soon as the state had identified which schools would get the new program, the researchers could randomly select schools for the control group. He did not realize that for the experiment to be valid, the state would first have to identify which schools were eligible for the new program, and then the researchers would randomly assign those eligible schools to either the treatment or the control group. Unfortunately, once this misconception was clarified, the official declined to participate, losing the opportunity to learn more about the effectiveness of the program and to set an example of statewide participation in a rigorous research study.