Unintended
Policy Impacts of Outcome-Based
Measures in a New Era of Service Delivery:
Examples from Preschool Immunization
Lynn
Olson and Andrew Gordon
Abstract
As public programs move to block grants and other more decentralized
forms of government intervention, one of the most vexing problems
is accountability. A commonly suggested solution is to develop an
outcome-based approach to measuring program success. One measure
that is already widely used to assess the quality of children's
health care is immunization rates among preschool children. This
paper examines the construction and use of immunization rates and
discusses the lessons learned as new performance standards are debated.
We probe the details of how group rates of immunizations are constructed,
and how these data are used in the process of evaluating quality
of care. Immunization rates seem highly appropriate for accountability:
they have strong face validity and are assumed to be a useful proxy
for good preventive care for children. But when the data and the
process of rate construction are examined closely, we find many
ambiguities. We suggest that while data point to a recent increase
in immunization rates, these higher rates do not necessarily indicate
that children's access to health care has improved. We will discuss
how recent efforts to increase immunization rates might have unintended
and even negative consequences for the well being of children. The
experience with measuring immunization rates underscores that performance
measures are inherently socially constructed facts, not perfect
mirrors of actions. Policy lessons include: 1) The resources and
attention devoted to a measure will change over time depending on
public and political interest; 2) even in the best designed projects,
critical information about how data are gathered remains mysterious;
3) even seemingly simple outcomes are rarely, if ever, simple to
measure; and 4) finally, and perhaps most important, documentation
requirements can corrupt the social processes the records are intended
to measure.
Lynn Olson, American Academy of
Pediatrics Andrew Gordon, Graduate School of Public Affairs,
University of Washington, Seattle
To Order:
Hard copies of IPR working papers cost $5.00 each (international orders are $10 each). We only accept checks drawn on U.S. bank and payable in U.S. funds. Checks or
money orders should be made payable to Northwestern University and sent to
the following address:
Publications Department - WP Orders
Institute for Policy Research
2040 Sheridan Rd., Evanston, IL 60208-4100.
For information, call 847-491-8712 or email ipr@northwestern.edu.
Please note that we do not accept credit cards.
