DNP865 Module 4 assignment: clinical decision support and alert fatigue analysis, a full sample

One in Five Patients Flagged: Clinical Decision Support, a Sepsis Prediction Alert, and the Cost of Alert Fatigue

Student Name

Doctor of Nursing Practice Program, Aspen University

DNP865: Healthcare Technologies and Informatics

Instructor Name

Month Day, Year

One in Five Patients Flagged: Clinical Decision Support, a Sepsis Prediction Alert, and the Cost of Alert Fatigue

Clinical decision support promises to bring the right information to the right person at the right time. When it fires too often or at the wrong people, it does the opposite: it trains clinicians to ignore it. This paper analyzes a proprietary sepsis prediction alert at a composite 350-bed hospital, uses external validation evidence and research on alert fatigue to explain its performance and costs, and recommends changes based on principles for effective decision support.

The Alert

Two years ago the hospital activated a sepsis prediction model supplied by its electronic health record vendor. The model calculates a score from vital signs, laboratory results, and other data every 15 minutes, and when the score crosses a threshold, it sends an interruptive alert to the patient's nurse and a notification to the provider, recommending a sepsis screen. Nurses report receiving several alerts per shift, most for patients they judge not septic. An audit of one month found that the alert fired for 19 percent of adult inpatients at least once, and that nurses dismissed it without documenting a sepsis screen 71 percent of the time.

What External Validation Shows

Proprietary models are often deployed without independent validation. In an external validation of the same widely implemented model at an academic health system, including 38,455 hospitalizations, the model had an area under the receiver operating characteristic curve of 0.63, well below the performance reported by its developer. It generated alerts for 18 percent of hospitalized patients, yet it failed to identify 67 percent of the patients who developed sepsis, and it identified only 183 of 2,552 septic patients who had not already received timely antibiotics (Wong et al., 2021). The model flagged many patients who were not septic, missed most who were, and added little that clinicians had not already noticed. The authors concluded that the model had poor discrimination and calibration and created a large burden of alert fatigue.

The Cost of Alert Fatigue

Alert fatigue is the declining responsiveness that follows exposure to frequent alerts, many of which are not useful. Its costs extend beyond the ignored alert itself. Pooled studies of medication warnings in electronic ordering show clinicians overriding most of them and trace the habit to error-producing conditions such as low specificity, unclear content, and unnecessary disruption of workflow (van der Sijs et al., 2006). In a study of primary care decision support, repeated alerts were common, and the likelihood of accepting a reminder dropped by 30 percent for each additional reminder received in an encounter (Ancker et al., 2017).

In the hospital, the costs include nurse time spent reading and dismissing alerts, interruptions during medication administration and other high-risk tasks, and erosion of trust that spills over to other alerts, including valuable ones. The most serious cost is that a genuinely septic patient's alert may be dismissed with the rest.

Principles for Effective Decision Support

A widely used implementer's guide judges any decision support intervention on five questions: is the content correct, is it going to the person who can act, is it presented in a usable form, is it delivered through a suitable channel, and does it arrive at a useful moment in the work (Osheroff et al., 2012). Measured against these, the sepsis alert fails on several counts. The information is often wrong, because of poor discrimination. It goes to the bedside nurse, who may not be the person best placed to act, and to the provider as a notification that is easily lost. Its format is an interruptive pop-up for a probabilistic score that does not explain why the patient was flagged. And it fires repeatedly every 15 minutes for the same patient.

Counting the Burden Locally

The audit allows a rough local estimate of the burden. With an average adult census of about 290 patients and the alert firing for 19 percent of them, roughly 55 patients trigger at least one alert each day, and repeat alerts for the same patients bring the daily total to about 140 interruptive alerts across the hospital. If each takes a nurse 45 seconds to read, consider, and dismiss, the alert consumes about 105 minutes of nursing time daily, or more than 600 hours a year, before counting the cost of interrupting other tasks. If local performance resembles the external validation, most of that time is spent on patients who are not septic, while most septic patients are identified by clinicians without the alert. These estimates will be refined with the hospital's own validation data, but even approximate numbers change the conversation with leadership from whether nurses dislike the alert to what it costs the organization and what it returns.

Recommendations

First, validate locally: before continuing to use the model, the informatics team should measure its sensitivity, positive predictive value, and alerts per true sepsis case using the hospital's own data, as the external study did. Second, retune the threshold and suppress repeat alerts for the same patient within a defined period. Third, change the recipient: route alerts to the rapid response nurse, who can review the patient's trend and decide whether to screen, rather than interrupting every bedside nurse. Fourth, change the format: display the factors driving the score so that clinicians can judge it, and use a non-interruptive worklist for lower scores. Fifth, measure: track alert volume per 100 patient-days, dismissal rates, time to sepsis screening and antibiotics, and sepsis mortality. If local validation confirms poor performance, the hospital should consider replacing the model with a simpler, transparent screening rule.

Conclusion

The sepsis alert illustrates both the promise and the hazard of clinical decision support. External validation shows the model performs far worse than expected, and research on alert fatigue shows how frequent, low-value alerts erode attention to all alerts. Applying the five rights and validating locally can turn an alert that nurses ignore into a signal worth acting on, or reveal that it should be retired.

References

Ancker, J. S., Edwards, A., Nosal, S., Hauser, D., Mauer, E., Kaushal, R., & HITEC Investigators. (2017). Effects of workload, work complexity, and repeated alerts on alert fatigue in a clinical decision support system. BMC Medical Informatics and Decision Making, 17, Article 36. https://doi.org/10.1186/s12911-017-0430-8

Osheroff, J. A., Teich, J. M., Levick, D., Saldana, L., Velasco, F. T., Sittig, D. F., Rogers, K. M., & Jenders, R. A. (2012). Improving outcomes with clinical decision support: An implementer's guide (2nd ed.). HIMSS.

van der Sijs, H., Aarts, J., Vulto, A., & Berg, M. (2006). Overriding of drug safety alerts in computerized physician order entry. Journal of the American Medical Informatics Association, 13(2), 138-147. https://doi.org/10.1197/jamia.M1809

Wong, A., Otles, E., Donnelly, J. P., Krumm, A., McCullough, J., DeTroyer-Cooley, O., Pestrue, J., Phillips, M., Konye, J., Penoza, C., Ghous, M., & Singh, K. (2021). External validation of a widely implemented proprietary sepsis prediction model in hospitalized patients. JAMA Internal Medicine, 181(8), 1065-1070. https://doi.org/10.1001/jamainternmed.2021.2626