Featured Articles

Survey Results: Smart Pump Data Analytics Pump Metrics that Should Be Monitored to Improve Safety

Between November 2017 and March 2018, ISMP conducted three surveys to learn about current practices with smart infusion pumps that have dose error-reduction software (DERS). Smart pumps can vastly improve medication safety by providing customizable libraries with dose limits and administration rates specific to medications and care areas. When the library is engaged, alerts can be generated when infusions are programmed outside these preset limits. The results of the first two surveys were published in our April 5, 2018 newsletter (Smart pumps in practice: Survey results reveal widespread use, but optimization is challenging). In this newsletter, we report on the results of the third survey, which focused on how pump data is being used to improve compliance and safety. Following the survey results, we offer a short discussion on pump analytics, a description of basic and advanced smart pump metrics to consider for review, and external resources available to assist you with smart pump analytics.

Survey Results

Respondent profile. ISMP thanks the 126 pharmacists (69%), nurses (19%), and others (12%) who completed the survey. Most (95%) respondents work in hospitals of varying bed size, with only 9% from hospitals with fewer than 100 beds, 52% from hospitals with 100-499 beds, and 36% from hospitals with 500 beds or more. Four out of every five respondents (81%) reported that they have direct participation in the review of available data from their organizations’ smart infusion pumps.

Frequency of data review. One in 10 respondents indicated that smart pump data is never analyzed in their organization. For most (60%) of these respondents, no data is available for review; for the others (40%), the data is available but never analyzed. For respondents who analyze pump data (n = 110), 52% review data quarterly and 26% review data monthly. Few respondents fall outside of the previously stated review periods, with only 6% reviewing data more often than monthly, and only 16% reviewing data less often than quarterly.

Resources and time. Almost all respondents (96%) reported that they believe reviewing pump data is essential to quality improvement. However, only about half of all respondents reported that their organization provides dedicated staff (56%) and dedicated time (48%) for reviewing pump data. More than half (59%) of the respondents reported spending less than 2 hours on pump data review each month. Another 34% of the respondents spend between 2-8 hours each month. Only 7% reported spending 9 hours or more on data review each month. Almost half (47%) of all respondents reported participating in a pump data analytics collaborative community or obtaining help from a pump vendor or external company to analyze their pump data.

Focus of analysis. Most organizations (95%) that analyze pump data focus on compliance with engaging the library and DERS. Approximately two-thirds of respondents assess alert frequency (64%) and reported that this is part of a larger alert fatigue reduction initiative (67%) in their organization. Only about half of the respondents analyze the action taken in response to an alert (51%) or use the data to investigate errors and adverse outcomes (48%) or to identify good catches (48%). Additional areas of focus reported in the “Other” category (12%) included interoperability compliance, improving clinical practices, and electronic health record (EHR) and library agreement.

Expertise and tools. Only 22% of respondents are fully confident that their organization has the necessary time, expertise, and tools to fully extract meaningful conclusions from smart pump data. Another 52% feel they are somewhat capable of data analysis. However, more than a quarter (26%) of respondents do not believe they have the requisite skills, tools, or time for meaningful data analysis.

Respondents who review pump data were most confident in their abilities to identify the top 10 medications by alert frequency (77% fully confident) and the number and type of overridden alerts (74% fully confident). They were less confident in their abilities to identify alerts due to programming infusions below minimum concentration limits (59% fully confident) and medications that have a low frequency of use but a high rate of alerts (39% fully confident). Approximately three-quarters of respondents (74%) are unable to identify risky practices associated with smart pump usage, such as unnecessary nurse dilution of products, which results in a nonstandard concentration.

Post-analysis actions. Respondents (83%) who analyze pump data reported that most of what is being learned is being used to update or change the drug library. Only 16% of respondents reported that data analytics has led to educational programs, and only 13% had made updates or changes in policies, procedures, and practices.

Challenges. We received close to 100 comments about the biggest challenges faced with analyzing smart pump data. Most of the challenges were associated with the usefulness of the pump data and the resources and expertise needed to extract and analyze meaningful information. For example, dozens of comments were either about not having enough pump data available for analysis or having too much data available for analysis. Many respondents also noted that the pump data available to them was not linked to individual patients, practitioners, units, or even hospitals if pump data reports were system-wide. Respondents also reported that data on basic infusions was unavailable, or that the analysis of pump data was not being shared with frontline staff. Challenges related to resources often described a lack of time, skill, and interest in pump data analytics, and undefined roles. Many respondents also noted that high compliance rates with engaging the library were misleading because many drugs were not built in the library and were being administered outside of the DERS. 

Discussion

Most US hospitals have invested in smart pumps with DERS. This technology has been on the market for more than 15 years; however, its safety benefits are not fully realized. Complacency and a low perception of risk associated with omitting certain drugs and fluids (solutions) from the library, failing to engage the library for available drugs and fluids, and overriding the DERS alerts, has permeated some healthcare facilities.

Smart pumps can capture extensive details about how they are being used and the alerts generated by this technology. For example, most smart pumps can capture the following data about each alert:1

  • Facility
  • Patient care unit or care profile
  • Drug or fluid
  • Device identifier
  • Infusion type
  • Programmed value
  • Hard or soft limit
  • Above or below a preset limit
  • Drug limits
  • Action taken in response to an alert
  • Library used
  • Time stamp
  • Times limit (degree to which the programmed value is over/under the library limit)
  • Other infusion- and limit-specific data (e.g., diluent volume, volume to be infused, drug amount, infusion rate, infusion duration, concentration)

Understanding, analyzing, and acting on this data is the primary way to maximize the remarkable safety benefits of this technology. Our survey respondents agree. Practically all believe that reviewing pump data is essential to improvement. However, many do not feel they have the dedicated staff, time, expertise, or tools to do this well. Those who review pump data frequently focus on library compliance and have acted on the data by adjusting the drug library and reducing nuisance alerts. While these attributes of pump use are important to safety, there is a wealth of untapped yet meaningful pump data that can be used to further improve medication safety.

Recommendations

To help organizations analyze, understand, and act on their smart pump data, we have provided a list of basic pump metrics that should be monitored at least quarterly by all organizations that use smart infusion pumps (Table 1). For organizations that are already conducting such analyses, we have also provided a list of advanced pump metrics to further improve medication safety (Table 2). We have also included some basic information on where you can find external help with smart pump data analytics.

Table 1. Basic smart pump metrics that should be monitored by all organizations using smart pumps

Metric

Rationale

Compliance with engaging the library (or interoperability between pumps and EHRs), by:

  • Facility
  • Patient care unit or care profile
  • Drug/fluid
  • Administering clinician (for interoperability users)
  • Improve overall compliance with engaging the library/interoperability
  • Improve compliance with engaging the library/interoperability in specific patient care units or care profiles
  • Identify trends in the drugs/fluids infused outside the library/interoperability so you can investigate why
  • Identify clinicians who may require additional training associated with interoperability  

Alerts, by:

  • Total number of alerts
  • Total number of alerts by patient care unit or care profile
  • Total number of alerts by drug/fluid
  • Total number of alerts by alert type (e.g., dose, concentration [including low concentration limits], duration, rate)
  • Total number of alerts by alert level (e.g., soft or hard, minimum or maximum)
  • Total number of alerts by infusion type (e.g., continuous, intermittent, primary, secondary, bolus, patient-controlled analgesia [PCA], epidural)
  • Total number of alerts by time of day
  • Total number of alerts by version of the drug library 
  • Identify and reduce nuisance alerts
  • Identify alert trends by patient care units or care profile, drugs/fluids, types of infusions (e.g., confusion between primary and secondary infusion; epidural infusions)
  • Make necessary changes to library limits
  • Detect trends after library changes to show   improvement/no change  
  • Identify trends associated with time of day (e.g., days, evenings)
  • Make necessary changes to how medications/fluids are prepared  
  • Set priorities for staff education
  • Identify and change risky practices (e.g., unnecessary bedside dilution may be the cause of a trend in low concentration alerts for a particular drug)

Action taken in response to an alert:

  • Percent of overridden alerts (number of overridden alerts/total number of actions taken in response to an alert)
  • Percent of cancelled alerts
  • Percent of alerts leading to reprogramming
  • Alerts overridden in less than 2 seconds
  • Identify the presence of alert fatigue and nuisance alerts
  • Assess the impact of the drug library limits
  • Identify trends by patient care unit, drugs/fluids
  • Identify and reduce error-prone programming
Table 2. Advanced smart pump metrics for monitoring and improvement

Metric

Rationale

Percent of all drugs/fluids available in the drug library (number of drug and fluid types available in the library/total number of drug and fluid types which are orderable in the EHR), by:

  • Drug/fluid
  • Ensure the library is available for most drugs/fluids administered via the pump
  • Improve the accuracy of library compliance rates
  • Increase reliability and trust in the technology

Library updates

  • Percent of pump libraries updated within 1 week
  • Identify delays in updating pump libraries within 1 week of issue

Alerts, by:

  • Rate of alerts (number of alerts/total number of infusions)
  • Rate of alerts by drug/fluid, patient care area or care profile, and clinician type (for interoperability users)
  • Further identify the presence and facilitate the reduction of alert fatigue and nuisance alerts
  • Provide increased ability to identify which medications need to be evaluated (e.g., medications which have low volume of use but a high alerting rate)
  • Monitor the impact of the interventions once implemented

Action taken in response to an alert:

  • Rate of overridden alerts (number of overridden alerts/total number of infusions)
  • Rate of cancelled alerts
  • Rate of alerts leading to reprogramming
  • Rate of alerted infusions ultimately reaching the patient (e.g., if an infusion is overridden and then cancelled or is overridden and then programmed as a basic infusion)
  • Rate of alerted infusions ultimately leading to basic infusions
  • Rate of hard stops leading to basic infusions
  • Provide increased ability to identify which medications need to be evaluated (e.g., medications which have low volume of use but a high alerting rate)
  • Establish if medications are definitively being administered within or outside the intended safety limits
  • Monitor the impact of interventions once implemented
  • Identify medication or fluid errors

Investigative reports, for:

  • Error or adverse drug event analysis
  • Trends with top drugs associated with alerts
  • Trends with failed autoprogramming attempts (for interoperability users) (number of failed autoprogramming attempts/total number of autoprogrammed infusions)
  • Understand the conditions associated with errors or adverse events
  • Understand the conditions associated with a drug/fluid alert trend (e.g., patient care units, infusion types, actions, programmed values) to determine whether to change library limits, educate staff, clarify a drug name to prevent confusion, or implement new practices1
  • Evaluate wireless connectivity and programming barriers

Library limits, compared to:

  • Electronic health records and electronic prescribing systems
  • Standardize drug names, concentrations, dosing units, dosing formulas, dose limits, concentration limits, and other drug information between the library, health record, and prescribing systems

External assistance with data analytics. Organizations can seek assistance with smart pump data analytics from external resources. First, different smart pump manufacturers offer data reports and analytic tools that vary in quality, quantity, and price. The organization is expected to conduct the analysis using the data reports provided by the manufacturer along with any available analytic tool. However, all data and tools provided by pump manufacturers are restricted to a single organization, without an opportunity for data sharing and collaborative learning. Contact your pump vendor for more information.

Another option for help with smart pump analytics is through membership in the Regenstrief National Center for Medical Device Informatics (REMEDI) Infusion Pump Collaborative. REMEDI is a vendor-neutral community of practice focused on smart pump technology and infusion therapy safety. REMEDI has compiled a database of more than 36 million alerts and compliance data representing almost 145 million infusions. Members have access to their own data and other hospitals’ data, including drug library details, alert data, and compliance data. REMEDI membership is currently provided at no cost to those willing to share their smart pump data and knowledge.

A final option known to ISMP is Bainbridge Health. Bainbridge Health provides software and clinical support services that can help hospitals achieve their medication safety goals while reducing the burden on employees to conduct pump data analysis. Their automated data analytics is combined with a team of clinicians who provide hospitals with detailed clinical interpretations, recommendations, and supportive benchmarking data to help them maximize their smart pump technology. This more hands-on approach to assistance with data analytics is provided for a fee.  

Reference: 1) Catlin AC, Malloy WX, Arthur KJ, et al. Comparative analytics of infusion pump data across multiple hospital systems. Am J Health Syst Pharm. 2015;72(4):317-24.