WP-07-14

Larry V. Hedges

Abstract

A common mistake in analysis of cluster randomized experiments is to ignore the effect of clustering and analyze the data as if each treatment group were a simple random sample. This typically leads to an overstatement of the precision of results and anti-conservative conclusions about precision and statistical significance of treatment effects. This paper gives a simple correction to the t-statistic that would be computed if clustering were (incorrectly) ignored in an experiment with two levels of nesting (e.g., classrooms and schools). The correction is a multiplicative factor depending on the number of clusters and subclusters, the subcluster sample size, the subcluster size, and the cluster and subcluster intraclass correlations ρS and ρC. The corrected t-statistic has Student’s t-distribution with reduced degrees of freedom. The corrected statistic reduces to the t-statistic computed by ignoring clustering when ρS = ρC = 0. It reduces to the t-statistic computed using cluster means when ρS = 1. If ρS and ρC are between 0 and 1, the adjusted t-statistic lies between these two and the degrees of freedom are in between those corresponding to these two extremes.

Larry V. Hedges, Board of Trustees Professor of Statistics and Social Policy, Northwestern University

Click on the working paper title at the top of this page to download a free pdf of the paper.* If You Need to Order a Hard Copy: Hard copies of IPR working papers cost $5.00 each (international orders are $10 each). We only accept checks drawn on U.S. bank and payable in U.S. funds. Checks or money orders should be made payable to Northwestern University and sent to the following address: Publications Department - WP Orders Institute for Policy Research 2040 Sheridan Rd., Evanston, IL 60208-4100. For information, call 847-491-8712 or email ipr@northwestern.edu. Please note that we do not accept credit cards. *Adobe Acrobat Reader 5.0 (or higher) is needed to read the Acrobat pdf. If you need to install Acrobat Reader, click the button below. Once the file has downloaded onto your desktop, run it to install the reader on your hard drive.