A small but growing number of people in the US are receiving messages from their doctors drafted with the help of artificial intelligence 鈥� and some may not even know it.
It is the first step in a larger plan to use OpenAI鈥檚 large language models 鈥� the line of technology powering chatbots such as ChatGPT 鈥� within one of the largest US electronic health records systems operated by the company Epic. Although Epic and healthcare organisations participating in the pilot deployment describe this as a low-risk use of large language models in healthcare, some AI researchers warn of challenges with the technology鈥檚 biases and tendencies to make up false information.
鈥淲e see this amazing technical achievement from large language models鈥� but I鈥檓 surprised that there aren鈥檛 more concerns about some of the specific use cases,鈥� says at the Massachusetts Institute of Technology.
Advertisement
More than two dozen physicians, nurses and other clinicians at three major healthcare organisations are among the first to test whether Epic鈥檚 AI-generated message drafts can speed up responses to patients while providing breathing room for healthcare teams. Research has found that clinicians can experience when messages fill their inboxes within an electronic health records system, and many answer such messages outside official work hours.
鈥淓ven if a [healthcare] provider can just get 30 seconds or 60 seconds back with a thought process, that adds up throughout the day,鈥� says at UW Health in Wisconsin, one of the organisations that has already begun pilot testing.
Such AI-written drafts can be used as is, edited by the clinician or discarded entirely. But a human is always responsible for manually clicking to send the message 鈥� a one-time alert within Epic鈥檚 system reminds the doctor to review and update the AI-generated draft reply before sending it out.
The AI-generated message feature is powered by OpenAI鈥檚 GPT-3 large language model. Each time it drafts a response, it uses information from the initial patient message and some of the patient鈥檚 medical history in the electronic health records system. Microsoft, which hosts GPT-3 in its data centres, said that it complies with US laws protecting and that any prompts or requests are never used to train or improve the core models owned by Microsoft or OpenAI such as GPT-3.
鈥淚f [AI] can draft these personalised, data-driven, well-written, empathetic responses to patients, then that should be able to decrease the mental effort and the time that it takes the caregiver to address the messages,鈥� says at Stanford Health Care in California, another participating organisation that plans to begin testing in May.
But not all patients who receive AI-generated messages through Epic鈥檚 system get alerted to how they were composed. Stanford Health Care and UW Health say they don鈥檛 have a system for notifying patients who receive AI-generated messages during the pilot deployments, although they have discussed that possibility.
The third participating organisation, UC San Diego Health in California, adds a disclaimer stating that part of the message was generated by AI in a secure environment and reviewed by the human sender. 鈥淲e鈥檙e in healthcare and we want to generate trust with our patients,鈥� says at UC San Diego Health. 鈥淎nd so we鈥檙e really erring on the side of disclosure.鈥�
Healthcare organisations should make it standard practice to tell patients up front, even if social expectations around such technology may change in the future, says at the University of Michigan. 鈥淭his is a little bit of a social experiment,鈥� says Singh.
Ghassemi also expressed concern about whether 鈥渙verworked and understaffed鈥� clinician teams will always carefully check AI-drafted messages for the factual errors that large language models are known to generate. She urged companies to address such issues before any real-world deployments, and pointed to previous AI tools deployed by Epic that in predicting the onset of sepsis in hospitalised patients or in predicting which patients may fail to show up for appointments.
Epic hasn鈥檛 publicly detailed how it evaluates AI performance. But the company said it will continue to update the AI-generated message feature with feedback from the participating healthcare organisations.
鈥淥ur goal in getting this out in a staged manner is to have organisations be able to study and publish on that accuracy directly within their system rather than through our internal testing and validation, as we think that鈥檒l provide more nuance to the conversation,鈥� says , senior vice president of research and development at Epic.
Epic鈥檚 plans for deploying more AI tools may soon face new regulatory requirements. The US government has requiring transparency for AIs deployed in electronic health records systems, says , president-elect of the American Medical Association. 鈥淭hese proposals are under review by the AMA, but appear to be a good first step towards responsible, ethical and transparent deployment of these tools,鈥� he says.
Article amended on 12 May 2023
We clarified that Microsoft hosts GPT-3 in its data centres
