CIS 450 Module 7 Evaluating an Information System Example

Does the Scanner Keep Patients Safe? Evaluating Bar-Code Medication Administration on One Unit

Student Name

Health Care Administration Program, Aspen University

CIS 450: Informatics in Healthcare

Instructor Name

Month Day, Year

Does the Scanner Keep Patients Safe? Evaluating Bar-Code Medication Administration on One Unit

Installing a health information system is only the beginning; evaluating whether it achieves its purpose is essential. This paper evaluates bar-code medication administration on a composite hospital's surgical unit one year after go-live, using a sociotechnical framework and several data sources, and recommends improvements.

The System

With bar-code administration, a nurse scans the wristband and then every dose before anything reaches the patient. The system checks the scan against the electronic medication administration record and alerts the nurse to the wrong patient, drug, dose, route or time. The hospital implemented it to reduce administration errors.

What the Evidence Predicts

A landmark study of bar-code medication administration with an electronic medication administration record in a large academic hospital found that nontiming administration errors dropped from about 12 to about 7 per 100 observed doses, and potential drug harms from those errors roughly halved (Poon et al., 2010). The unit's leaders hoped for similar gains.

Evaluation Framework

Sittig and Singh's eight-part sociotechnical model gave the evaluation its structure, covering technology, content, interface, staff, workflow, internal policy, outside rules and measurement (Sittig & Singh, 2010). This framework directs attention beyond whether the scanner works to whether the whole system supports safe administration.

Data Sources

The evaluator reviewed scan compliance reports for 12 months, observed 200 medication administrations across shifts, analyzed override reasons, surveyed 32 unit nurses and reviewed medication error reports before and after go-live.

Findings

The table summarizes key findings by dimension.

Observed Errors

Among 200 observed administrations, the evaluator noted 9 nontiming errors, a rate of 4.5%, lower than the pre-implementation audit's 10% but with three errors occurring when medications were scanned after, rather than before, administration. The unit's error reports showed fewer wrong-patient and wrong-dose events than the year before go-live.

Workarounds

Nurses described workarounds: scanning a barcode taped to the medication cart instead of the patient's wristband when the wristband was hard to reach, and scanning after giving medications during busy periods. Workarounds usually arise from system and workflow problems, such as failing scanners and dead zones, and they undermine the safety the system is meant to provide.

Acceptance

Survey results showed that 78% of nurses believed the system improved safety, but only 44% found it easy to use during busy periods. The technology acceptance model helps explain the pattern: nurses saw the system as useful but not easy, and difficulty drove workarounds (Holden & Karsh, 2010).

Comparison With the Trial

The unit's error reduction was meaningful but smaller than in the landmark trial. Differences likely reflect lower medication scan compliance, hardware problems and workarounds. The comparison suggests that the unit has not yet achieved the system's full potential.

Recommendations

The evaluator recommended replacing failing scanners and fixing the wireless dead zone; working with pharmacy to barcode all dispensed medications; tuning early-dose alerts; clarifying the override policy and requiring a reason for each override; training night and float staff; posting monthly scan compliance by shift; and redesigning medication preparation to one patient at a time.

Follow-Up Plan

The unit will repeat observations and surveys six months after changes, targeting 95% medication scan compliance and fewer observed nontiming errors. Monthly compliance reports will continue, and override reasons will be reviewed quarterly.

Lessons for Evaluation

Evaluating a system requires several data sources: automated reports show compliance but not why it falls short; observation reveals workarounds; surveys capture perceptions; error reports show outcomes. A sociotechnical framework ensures that technical, human and organizational factors are all examined.

Costs of the System

The evaluation also considered costs: scanners, wireless upgrades, pharmacy barcoding, training and informatics time. Estimating the costs of preventable adverse drug events avoided helped leaders see that fixing the unit's problems was worth the investment. Evaluations that include costs are more persuasive to administrators.

Engaging Front-Line Staff

The evaluator shared preliminary findings with nurses before finalizing recommendations. Nurses added that barcode labels on some intravenous bags smudged, which explained scan failures the reports had not. Involving the people who use the system improves both the accuracy of the evaluation and support for changes.

Patient Perspective

Patients noticed the scanning. Some found it reassuring; a few found the wristband scans during sleep disruptive. The unit began bundling night medications to reduce awakenings and explained the purpose of scanning at admission, turning a safety process into a visible sign of care.

Data Limitations

Each data source had limits. Compliance reports count scans but cannot show whether a scan preceded administration. Observation may change behavior while observers are present. Surveys reflect perception, and error reports undercount events. Combining sources compensates for these limits, and the report states them plainly.

Linking to Organizational Goals

The evaluation tied its recommendations to the hospital's medication safety goals and accreditation priorities. Framing technical fixes in terms of organizational goals helped the evaluator secure budget for scanners and pharmacy labeling that might otherwise have waited.

Timing of Evaluation

Evaluating one year after go-live allowed initial learning curves to settle while habits were still changeable. Evaluations too early may measure inexperience, and evaluations too late may find workarounds deeply entrenched.

Sharing Results

The evaluator presented findings to the unit, the nursing leadership council and the medication safety committee, tailoring each presentation to the audience's decisions. Sharing results widely built support and let other units apply the lessons before their own problems grew.

Conclusion

One year after go-live, bar-code medication administration had reduced errors on the surgical unit but less than a landmark trial achieved. A sociotechnical evaluation traced the gap to hardware failures, unbarcoded medications, alert fatigue, training gaps, workflow habits and unclear policy, which produced workarounds. Targeted fixes and ongoing measurement offer a path to the system's full safety benefit.

References

Holden, R. J., & Karsh, B.-T. (2010). The technology acceptance model: Its past and its future in health care. Journal of Biomedical Informatics, 43(1), 159-172. https://doi.org/10.1016/j.jbi.2009.07.002

Poon, E. G., Keohane, C. A., Yoon, C. S., Ditmore, M., Bane, A., Levtzion-Korach, O., Moniz, T., Rothschild, J. M., Kachalia, A. B., Hayes, J., Churchill, W. W., Lipsitz, S., Whittemore, A. D., Bates, D. W., & Gandhi, T. K. (2010). Effect of bar-code technology on the safety of medication administration. New England Journal of Medicine, 362(18), 1698-1707. https://doi.org/10.1056/NEJMsa0907115

Sittig, D. F., & Singh, H. (2010). A new sociotechnical model for studying health information technology in complex adaptive healthcare systems. Quality and Safety in Health Care, 19(Suppl. 3), i68-i74. https://doi.org/10.1136/qshc.2010.042085