Experimental Political Science
IPR fellow leads conference for first handbook on the topic
On May 28 and 29, more than 100 political scientists and graduate students from across the country gathered at Northwestern University to discuss and critique papers that will be included as chapters in the first "Handbook of Experimental Political Science."
The participant list read like a who's who of pioneers of experimental methods in the field—Shanto Iyengar and Paul Sniderman of Stanford University, Alan Gerber and Donald Green of Yale University, Diana Mutz of the University of Pennsylvania, and Arthur Lupia of the University of Michigan, to name a few.
IPR political scientist James Druckman was the conference organizer. As one of the handbook's co-editors and the first political scientist named as a lead editor of Public Opinion Quarterly, he is leading the charge to improve methodology in the field.
Druckman pointed out why Northwestern made such a great intellectual venue for the conference. "NU is where the father of quasi-experimentation, Donald Campbell, spent most of his career, and where then-graduate student Margaret Hermann used role-playing simulations to model the events that led to World War I-research that has been highly influential in the field," he said. He also noted the contributions of Daniel Druckman, his father, who played an important role in the early development of experiments at Northwestern, working with Campbell and Harold Guetzkow to explore international negotiations. Hermann is a professor at Syracuse and Daniel Druckman is a professor at Geroge Mason University; both are contributors to the volume.
Fast forward several decades, IPR is still a vibrant intellectual center for research and training on experimental methods. Social psychologist Thomas D. Cook, Campbell's longtime collaborator and a leading authority in his own right, continues to unravel questions of quasi-experimentation, particularly in the field of education. The Institute houses the Center for Improving Methods for Quantitative Policy Research, or Q-Center, led by statistician and education researcher Larry Hedges, an international authority on meta-analysis. Sociologist Jeremy Freese is leading a revamp of the online project Time-sharing Experiments for the Social Sciences, or TESS, that Mutz and Lupia launched in 2002, and for which Druckman is an associate PI.
"In fact, more than eight Northwestern alumni or faculty are involved in the volume," Druckman said.
Many at the conference marveled at how the trend in using experimentation has shifted. Today, it has become more acceptable—even "fashionable"—for papers to feature experimental methods. This stands in stark contrast to years, in the not too distant past, where reviewers often dismissed journal submissions outright for inclusion of such methods.
"We are no longer the lunatic fringe," noted Don Kinder of the University of Michigan.
Still, many nagging questions about experiments remain to be tackled. Thus, the conference participants held a two-day "running conversation" on some of the pressing issues of experimental methods, covering not only the nuts and bolts of running experiments—limitations of internal and external validity, causality, design and analytic changes—but also specific considerations for experiments measuring particular phenomena, such as vote choice, elite bargaining, and political attitudes.
For example, in a discussion of monetary and other types of compensation for experimental subjects, Eric Dickson of New York University defended the use of incentives, suggesting that, at least in some cases, they can tell the researchers more than might be learned otherwise. While researchers should beware the possibility of biasing a sample, he noted that well-planned incentives can draw in a specific subgroup that otherwise would not be reached, rounding out the sample’s representativeness.
Kinder wrapped up the exhaustive two-day overview of political science experimentation with some general comments for the chapter authors. He urged his colleagues to carefully weigh which experiments to feature and avoid the fatal trap of trying to be too inclusive.
"For any particular area of research, the more experiments included, the higher the chance for confusion on the reader's part," Kinder pointed out. "Literature reviews that try to cover everything are dangerous."
Kinder suggested that the researchers concentrate on a few exemplary experiments that reveal unexpected findings. He also recommended that they try to be "embracingly ecumenical in spirit" and expand their discussions to include a variety of different methods, including field experiments, lab experiments, and observational studies.
"What can we learn from various forms of social science research?" Kinder queried.
Druckman was pleased with the conference proceedings. "We more than fulfilled the basic goals set out for the conference—to situate experiments in the broader discipline, present important methodological issues, and review the contributions of experimental research," he said. "And now we are well set to meet the challenge of combining everything into a unified volume."
Druckman is associate professor of political science and an IPR faculty fellow. "The Handbook of Experimental Political Science" will be published by Cambridge University Press.
|
