Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational Interventions

Howard S. Bloom; Lashawn Richburg-Hayes; Alison Rebeck Black

doi:10.3102/0162373707299550

journal article Mar 01, 2007

Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational Interventions

Howard S. Bloom Lashawn Richburg-Hayes Alison Rebeck Black

Educational Evaluation and Policy Analysis Vol. 29 No. 1 pp. 30-59 · American Educational Research Association (AERA)

View at Publisher Save 10.3102/0162373707299550

Abstract

This article examines how controlling statistically for baseline covariates, especially pretests, improves the precision of studies that randomize schools to measure the impacts of educational interventions on student achievement. Empirical findings from five urban school districts indicate that (1) pretests can reduce the number of randomized schools needed for a given level of precision to about half of what would be needed otherwise for elementary schools, one fifth for middle schools, and one tenth for high schools, and (2) school-level pretests are as effective in this regard as student-level pretests. Furthermore, the precision-enhancing power of pretests (3) declines only slightly as the number of years between the pretest and posttests increases; (4) improves only slightly with pretests for more than 1 baseline year; and (5) is substantial, even when the pretest differs from the posttest. The article compares these findings with past research and presents an approach for quantifying their uncertainty.

Topics

No keywords indexed for this article. Browse by subject →

References

28

[1]

10.1177/0193841x9501900504

[2]

Bloom HS (2005)

[3]

10.1177/0193841x9902300405

[4]

Bloom HS (2006)

[5]

BloomHSRichburg-HayesLBlackARUsing covariates to improve precision: Empirical guidance for studies that randomize schools to measure the impacts of educational interventions2005NovemberNew YorkMDRCAvailable athttp://www.mdrc.org/publications/417/full.pdf

[6]

Cohen J (1977)

[7]

Donner A (2000)

[8]

Feng Z Statistics in Medicine (1999) 10.1002/(sici)1097-0258(19990315)18:5<539::aid-sim50>3.0.co;2-s

[9]

Fisher RA (1947)

[10]

Frison L Statistics in Medicine (1992) 10.1002/sim.4780111304

[11]

Gargani J (2005)

[12]

Intraclass Correlation Values for Planning Group-Randomized Trials in Education

Larry V. Hedges, E. C. Hedberg

Educational Evaluation and Policy Analysis 10.3102/0162373707299706

[13]

10.1177/1090198104263406

[14]

Kane T (2004)

[15]

Lipsey MW (1990)

[16]

Martin DC Statistics in Medicine (1993) 10.1002/sim.4780120315

[17]

Murray DM (1998)

[18]

10.1177/0193841x02239019

[19]

Murray D Journal of Studies on Alcohol (1995) 10.15288/jsa.1995.56.681

[20]

National Center for Education Statistics (1997)

[21]

10.3102/01623737021002127

[22]

Statistical analysis and optimal design for cluster randomized trials.

Stephen W. Raudenbush

Psychological Methods 1997 10.1037/1082-989x.2.2.173

[23]

Raudenbush SW Psychological Methods (2001) 10.1037/1082-989x.6.4.387

[24]

10.3102/0162373707299460

[25]

Schochet PA (2005)

[26]

Siddiqui O American Journal of Epidemiology (1996) 10.1093/oxfordjournals.aje.a008945

[27]

Singer JD (2003) 10.1093/acprof:oso/9780195152968.001.0001

[28]

Ukoumunne OC Health Technology Assessment (1999) 10.3310/hta3050

Cited By

181

Intervention for First Graders With Limited Number Knowledge

Russell Gersten, Eric Rolfhus · 2015

American Educational Research Journ...

Reference Values of Within-District Intraclass Correlations of Academic Achievement by District Characteristics

E. C. Hedberg, Larry V. Hedges · 2014

Evaluation Review

PowerUp!: A Tool for Calculating Minimum Detectable Effect Sizes and Minimum Required Sample Sizes for Experimental and Quasi-Experimental Design Studies

Nianbo Dong, Rebecca Maynard · 2013

Journal of Research on Educational...

Metrics

181

Citations

28

References

Details

Published: Mar 01, 2007
Vol/Issue: 29(1)
Pages: 30-59
License: View

Authors

H

Howard S. Bloom

MDRC

L

Lashawn Richburg-Hayes

MDRC

A

Alison Rebeck Black

MDRC

Cite This Article

Howard S. Bloom, Lashawn Richburg-Hayes, Alison Rebeck Black (2007). Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational Interventions. Educational Evaluation and Policy Analysis, 29(1), 30-59. https://doi.org/10.3102/0162373707299550

Using Covariates to Improve Precision for Studies That Randomize Schools to Evaluate Educational Interventions

You May Also Like