Benchmarking - When is it Dangerous?

At first glance, medication error rates may seem ideal for benchmarking. Yet, we must question the wisdom of applying the benchmarking concept to the medication use process when the focus is on error rates. The true incidence of medication errors varies, depending heavily on the rigor with which the events are identified and reported..

Certainly, the confusion surrounding the term "benchmarking" perpetuates the myth that one can gauge the quality and safety of the medication use process simply by comparing error rates, both within an organization and externally. Benchmarking is an ongoing process that determines how other organizations have achieved the best performance and suggests ways for adapting the best practices that result in this exceptional performance. Although measurement is one of its components, effective benchmarking is a dual process that requires two products: benchmarks and enablers.¹ Benchmarks are measures of comparative performance that answer the question: "What is your level of performance?" Alone, this information has little use in improving performance. Benchmarking must also provide a systematic method of understanding the underlying processes that determine organizational performance. To that end, enablers must be identified. Enablers are the specific practices that lead to exemplary performance and answer the question: "How do you do it?" Overlooking either one of these components in the benchmarking process renders it useless, even dangerous!

Currently, there is no consistent process among healthcare organizations for detecting and reporting errors. Since many medication errors cause no harm to patients, they remain undetected or unreported. Still, organizations frequently depend on spontaneous voluntary error reports alone to determine a medication error rate. The inherent variability of determining an error rate in this way invalidates the measurement, or benchmark. A high error rate may suggest either unsafe medication practices or an organizational culture that promotes error reporting. Conversely, a low error rate may suggest either successful error prevention strategies or a punitive culture that inhibits error reporting. Also, the definition of a medication error may not be consistent among organizations or even between individual practitioners in the same organization. Thus, spontaneous error reporting is a poor method of gathering "benchmarks;"it is not designed to measure medication error rates.

Of equal concern is the mistaken belief that benchmarking is simply comparing numbers.² Although not meaningful, healthcare organizations have embraced the practice of comparing error rates. Yet, there has been little effective effort directed at identifying enablers for safe medication use to accompany this attempt at benchmarking. As a result, organizations focus undue attention on maintaining a low error rate, giving the errors themselves, rather than their correction, disproportionate importance. This promotes an unproductive cycle of underreporting errors, which results in unrecognized weaknesses in the medication use system. Thus, low error rates can result in a false sense of security and a tacit acceptance of preventable errors.

Benchmarking for the medication use process can be effective only if a system of objective measurement, more reliable than spontaneous error reporting alone, is used to identify best practices (such as observational methods or systematic evaluation of errors ^3,4 ). In addition, the benchmarking process must include a method for accurately determining the specific processes that enable the organization to achieve an environment where medications are safely used. Success is more likely with benchmarking projects that are focused on specific areas of drug therapy (such as insulin therapy or anticoagulant therapy) so that accurate benchmarks (performance measurements) and enablers (practices that lead to exemplary performance) can be more easily identified and implemented. So, select your benchmarking projects carefully. Meanwhile, we urge organizations to place less emphasis on error rates based solely on spontaneous voluntary reporting programs. Instead, encourage error reporting to identify and remedy problems, not to provide statistics for comparison.

References:

1. Dinklage K. Learning from the best: using benchmarking to improveperformance. Pharmaguide to Hospital Medicine. 1994; 7(3):5-8.
2. ASHRM (US). Health care risk management benchmarking primer. Chicago: AHA;1996.
3. Allen EL, Barker KN. Fundamentals of medication error research. AM J Hosp Pharm. 1990;47:555-71.
4. Lesar TS. Factors related to errors in medication prescribing. JAMA. 1997; 277 (4):312-317.