WP-07-02

Thomas Cook and Vivian Wong

Abstract

This paper reviews the literature on whether regression-discontinuity studies reproduce
the results of randomized experiments conducted on the same topic. After briefly reviewing the regression-discontinuity design and its history, we explicate the general conditions necessary for a strong test of correspondence in results when an experiment is used to validate any non-experimental method. In Economics, within-study comparisons of this kind are associated with LaLonde (1986), and we elaborate on how to do such studies better than twenty years ago. We identify three cases where regression discontinuity and experimental results with overlapping samples were explicitly contrasted. By criteria of both effect sizes and statistical significance patterns, we then show that each study produced similar results across the experiment and regression discontinuity study. This correspondence is what theory predicts. But to achieve it in the complex social settings in which these within-study comparisons were carried out suggests that regression discontinuity results may be generally robust.

Thomas Cook, Sociology, Psychology, Education and Social Policy
Vivian C. Wong, Human Development and Social Policy

Click on the working paper title at the top of this page to download a free pdf of the paper.* If You Need to Order a Hard Copy: Hard copies of IPR working papers cost $5.00 each (international orders are $10 each). We only accept checks drawn on U.S. bank and payable in U.S. funds. Checks or money orders should be made payable to Northwestern University and sent to the following address: Publications Department - WP Orders Institute for Policy Research 2040 Sheridan Rd., Evanston, IL 60208-4100. For information, call 847-491-8712 or email ipr@northwestern.edu. Please note that we do not accept credit cards. *Adobe Acrobat Reader 5.0 (or higher) is needed to read the Acrobat pdf. If you need to install Acrobat Reader, click the button below. Once the file has downloaded onto your desktop, run it to install the reader on your hard drive.