CIS 450 Module 8 An Emerging Technology in Health Informatics Example

A Draft Before You Reply: Generative AI for Patient Messages as an Emerging Informatics Technology

Student Name

Health Care Administration Program, Aspen University

CIS 450: Informatics in Healthcare

Instructor Name

Month Day, Year

A Draft Before You Reply: Generative AI for Patient Messages as an Emerging Informatics Technology

Generative artificial intelligence, built on large language models that produce text, is spreading quickly into health care. One of the earliest uses is drafting replies to patient messages in the electronic record inbox, where message volume has grown sharply. This paper reviews that technology, the early evidence, its risks and a phased adoption recommendation for a composite health system.

How It Works

Each incoming portal message, with selected details from the chart, goes to a large language model that writes a suggested answer. The clinician sees the draft beside the original message and can edit, use or discard it. The model does not send anything on its own. Drafts can be tailored by message type, such as medication questions or test result questions.

What Early Evidence Shows

Evidence is early but informative. In a study comparing physician and chatbot responses to patient questions posted on a public social media forum, evaluators preferred the chatbot's responses in 78.6% of evaluations, rating them higher in quality and empathy; the chatbot's answers were also much longer (Ayers et al., 2023). The setting was a public forum, not a clinic inbox, so the findings suggest potential rather than proven benefit.

A Real Inbox Pilot

A five-week quality improvement pilot at an academic medical center tested record-integrated drafts for 162 clinicians. The mean rate of using AI-generated drafts was 20%; there was no change in time spent reading, writing or acting on replies, but clinicians' task load and work exhaustion scores fell significantly (Garcia et al., 2024). The technology eased perceived burden more than it saved measurable time.

Readiness Assessment

The composite health system assessed its readiness along several dimensions.

Risks: Accuracy

Language models can produce confident but wrong statements, such as incorrect medication advice or invented test details. Drafts are only as safe as the clinician's review, and busy clinicians may accept fluent drafts without careful reading. The system would require clinicians to review every draft and would audit samples for errors.

Risks: Bias and Equity

Algorithms can reproduce inequities. A study of a widely used commercial risk algorithm showed that predicting cost rather than illness led to substantial racial bias in who received extra care (Obermeyer et al., 2019). Language models could also respond differently to messages written in different dialects or languages. The system would test draft quality across languages and patient groups.

Risks: Privacy and Security

Messages contain protected health information. The model must run within an environment covered by appropriate agreements and security controls, and the vendor must not use patient data to train models outside those agreements. The system's privacy officer would review data flows before any pilot.

Transparency With Patients

Patients may want to know whether a reply was drafted by AI. The system considered disclosure options, such as a note that replies may be drafted with assistance and are reviewed by the care team. Transparency supports trust, though its effect on patient perceptions requires study.

Effects on Work

The pilot evidence suggests drafts may reduce cognitive burden even without saving time. For staff roles, drafts might help nurses and medical assistants answer routine questions consistently. Administrators must ensure that responsibility for content remains clear and that drafts do not quietly shift clinical judgment to less-trained staff.

Recommendation

The system should adopt the technology in phases: first, create an AI governance committee and policy; second, pilot drafts for one department's routine message types with volunteer clinicians; third, audit a sample of sent replies weekly for accuracy, tone and equity; fourth, measure use, time, clinician burden and patient feedback; and fifth, expand only if accuracy and satisfaction targets are met.

Keeping Up

Generative AI changes rapidly; model versions and vendor features evolve monthly. Evaluation cannot be a one-time event. The governance committee should review performance regularly and reassess when models change.

Measuring Value

Value should be measured, not assumed. The pilot will track the share of messages using drafts, edits made to drafts, time spent in the inbox, clinician burden scores, patient satisfaction with replies and any safety events. Because the academic pilot found burden reductions without time savings, the system will judge value on both.

Other Generative AI Uses

Drafting messages is one of several generative AI applications emerging in health care, alongside ambient visit documentation, summarizing long records and drafting prior authorization letters. Each raises similar questions about accuracy, oversight, privacy and equity. A governance structure built for inbox drafts can guide evaluation of later uses.

Training Clinicians

Clinicians need training not only in using the tool but in reviewing its output critically: checking facts against the record, watching for tone that does not fit the patient, and editing rather than accepting by default. Training should include examples of plausible but wrong drafts so clinicians learn what to look for.

Vendor Questions

Before any pilot, the governance committee will ask the vendor which model is used, how it was tested for accuracy and bias, whether patient data are used to train it, how updates are communicated and how errors can be reported. Clear answers are a condition of adoption.

Patient Voice

The system will ask its patient advisory council to review sample AI-assisted replies and the disclosure language before launch. Patients can judge tone and clarity in ways clinicians may miss, and their involvement signals respect for the people the technology is meant to serve.

Exit Plan

Adoption should include a way out. If audits reveal persistent errors or patient complaints, the system will pause drafts and revert to standard replies without disrupting care. Planning an exit in advance makes it easier to act if evidence turns against the technology.

Conclusion

Generative AI drafts for patient messages address a real problem, overflowing inboxes, and early evidence suggests reduced burden and well-received responses, though not yet time savings. Risks of inaccuracy, bias and privacy require governance, clinician review, audits and transparency. A phased, measured adoption lets a health system learn whether the technology helps its own clinicians and patients.

References

Ayers, J. W., Poliak, A., Dredze, M., Leas, E. C., Zhu, Z., Kelley, J. B., Faix, D. J., Goodman, A. M., Longhurst, C. A., Hogarth, M., & Smith, D. M. (2023). Comparing physician and artificial intelligence chatbot responses to patient questions posted to a public social media forum. JAMA Internal Medicine, 183(6), 589-596. https://doi.org/10.1001/jamainternmed.2023.1838

Garcia, P., Ma, S. P., Shah, S., Smith, M., Jeong, Y., Devon-Sand, A., Tai-Seale, M., Takazawa, K., Clutter, D., Vogt, K., Lugtu, C., Rojo, M., Lin, S., Shanafelt, T., Pfeffer, M. A., & Sharp, C. (2024). Artificial intelligence-generated draft replies to patient inbox messages. JAMA Network Open, 7(3), Article e243201. https://doi.org/10.1001/jamanetworkopen.2024.3201

Obermeyer, Z., Powers, B., Vogeli, C., & Mullainathan, S. (2019). Dissecting racial bias in an algorithm used to manage the health of populations. Science, 366(6464), 447-453. https://doi.org/10.1126/science.aax2342