Benchmarking - when is it dangerous?
From the September 9, 1998 issue
At first glance, medication error rates may seem ideal for
benchmarking. Yet, we must question the wisdom of applying
the benchmarking concept to the medication use process when
the focus is on error rates. The true incidence of medication
errors varies, depending heavily on the rigor with which the
events are identified and reported..
Certainly, the confusion surrounding the term "benchmarking"perpetuates
the myth that one can gauge the quality and safety of the
medication use process simply by comparing error rates, both
within an organization and externally. Benchmarking is an
ongoing process that determines how other organizations have
achieved the best performance and suggests ways for adapting
the best practices that result in this exceptional performance.
Although measurement is one of its components, effective benchmarking
is a dual process that requires two products: benchmarks and
enablers.1 Benchmarks are measures of comparative
performance that answer the question: "What is your level
of performance?" Alone, this information has little use in
improving performance. Benchmarking must also provide a systematic
method of understanding the underlying processes that determine
organizational performance. To that end, enablers must be
identified. Enablers are the specific practices that lead
to exemplary performance and answer the question: "How do
you do it?" Overlooking either one of these components in
the benchmarking process renders it useless, even dangerous!
Currently, there is no consistent process among healthcare
organizations for detecting and reporting errors. Since many
medication errors cause no harm to patients, they remain undetected
or unreported. Still, organizations frequently depend on spontaneous
voluntary error reports alone to determine a medication error
rate. The inherent variability of determining an error rate
in this way invalidates the measurement, or benchmark. A high
error rate may suggest either unsafe medication practices
or an organizational culture that promotes error reporting.
Conversely, a low error rate may suggest either successful
error prevention strategies or a punitive culture that inhibits
error reporting. Also, the definition of a medication error
may not be consistent among organizations or even between
individual practitioners in the same organization. Thus, spontaneous
error reporting is a poor method of gathering "benchmarks;"it
is not designed to measure medication error rates.
Of equal concern is the mistaken belief that benchmarking
is simply comparing numbers.2 Although not meaningful,
healthcare organizations have embraced the practice of comparing
error rates. Yet, there has been little effective effort directed
at identifying enablers for safe medication use to accompany
this attempt at benchmarking. As a result, organizations focus
undue attention on maintaining a low error rate, giving the
errors themselves, rather than their correction, disproportionate
importance. This promotes an unproductive cycle of underreporting
errors, which results in unrecognized weaknesses in the medication
use system. Thus, low error rates can result in a false sense
of security and a tacit acceptance of preventable errors.
Benchmarking for the medication use process can be effective
only if a system of objective measurement, more reliable than
spontaneous error reporting alone, is used to identify best
practices (such as observational methods or systematic evaluation
of errors 3,4 ). In addition, the benchmarking
process must include a method for accurately determining the
specific processes that enable the organization to achieve
an environment where medications are safely used. Success
is more likely with benchmarking projects that are focused
on specific areas of drug therapy (such as insulin therapy
or anticoagulant therapy) so that accurate benchmarks (performance
measurements) and enablers (practices that lead to exemplary
performance) can be more easily identified and implemented.
So, select your benchmarking projects carefully. Meanwhile,
we urge organizations to place less emphasis on error rates
based solely on spontaneous voluntary reporting programs.
Instead, encourage error reporting to identify and remedy
problems, not to provide statistics for comparison.
1) Dinklage K. Learning from the best: using benchmarking
to improveperformance. Pharmaguide to Hospital Medicine. 1994;
2) ASHRM (US). Health care risk management benchmarking primer.
3) Allen EL, Barker KN. Fundamentals of medication error research.
AM J Hosp Pharm. 1990;47:555-71.
4) Lesar TS. Factors related to errors in medication prescribing.
JAMA. 1997; 277 (4):312-317.