Many court cases are decided by juries, and it has
long been known that juries sometimes make incorrect decisions.
The accuracy of jury verdicts can be studied empirically and systematically.
In the field of sample surveys, it is common to estimate sampling
accuracy even though the true value is not observed—the
key is to use replication. A similar approach can be used in methodology
to estimate the accuracy of jury verdicts when “replications”
such as second decisions on the cases are available. A recent
study by the National Center for State Courts (NCSC) collected
such replication data for criminal cases in four large U.S. metropolitan
areas between 2000 and 2001. The levels of agreement between the
judge’s verdict and jury’s verdict were only fair
to poor, remarkably similar to the Kalven and Zeisel data collected
in the 1950s. Estimates of jury accuracy can be developed from
the replication data, although stronger assumptions are needed
than for random sample surveys. Such assumptions are identified
and the effects of failure of the assumptions are analyzed. Under
some plausible conditions, the estimates tend to overstate jury
accuracy.
Finer analyses of jury accuracy can be obtained
with stronger models. Log-linear statistical models were fitted
to the data to provide direct estimates of the differential accuracy
between judge and jury. The models exploit a novel feature of
the NCSC data: The inclusion of evaluations of the strength of
evidence in the cases. The models are subject to error in specification,
including the assumption of independence between the judge and
the jury as well as imperfect measurement of strength of evidence.
Thus, the estimates based on the model should be interpreted cautiously.
The estimates from the model support the views that (1) juries
were more likely than judges to acquit when the defendant was
not guilty, and (2) judges were more likely than juries to convict
when the defendant was guilty. The estimates do not support the
view that the jury’s probability of convicting a not-guilty
defendant is smaller than its probability of acquitting a guilty
defendant. These findings, even taken with due caution, should
not be generalized outside the NCSC study. Future replications
of the NCSC study would be valuable and should be carried out
over time and in other jurisdictions.
Bruce D. Spencer, Professor
of Statistics, Northwestern University
Click on
the working paper title at the top of this
page to download a free pdf of the paper.*
If You Need to Order a Hard Copy: Hard copies
of IPR working papers cost $5.00 each (international orders are $10 each). We only accept checks drawn on U.S. bank and payable in U.S. funds. Checks or money orders should be made
payable to Northwestern University and sent to the following address:
Publications Department - WP Orders
Institute for Policy Research
2040 Sheridan Rd., Evanston, IL 60208-4100.
For information, call 847-491-8712 or email ipr@northwestern.edu.
Please note that we do not accept credit cards.
*Adobe Acrobat Reader 5.0 (or higher) is needed to read the Acrobat pdf.
If you need to install Acrobat Reader, click the button below. Once the file
has downloaded onto your desktop, run it to install the reader on your hard
drive.
