Will the toothpick I swallowed kill me? How big a deal is the lump I got on my head after running into a metal bar?
Sometime soon these questions may be answered by software rather than doctors — and no offense doc, but the software’s responses may be smarter and have a better bedside manner.
That’s the conclusion of a just-released study of a new artificial intelligence model called ChatGPT by researchers from the Johns Hopkins University and the University of California San Diego.
“People have been playing around with ChatGPT and finding all sorts of new capabilities in the medical fields,” said Mark Dredze, an associate professor of computer science in Hopkins’ Whiting School of Engineering and a co-author of the study just published in the academic journal JAMA Internal Medicine.
“That doesn’t mean the results are good, so we did a study to see how good they are really,” he said. “Turns out the results are pretty good.”
Since its launch in November, provided for free by the research and technology company OpenAI for basic use, ChatGPT has captured a lot of public attention for its ability to write human-like text. It can write term papers for students or, Dredze said, turn JAMA studies into a haiku.
It also can do a solid job explaining the potential repercussions of ingesting a small sliver of wood and not making the swallower feel bad about it.
There may be particular interest in using the software in health care. Patients became far more comfortable with technology to communicate with their doctors during the coronavirus pandemic, Dredze said. Many hospitals and practices also have turned to messaging platforms that allow patients to ask questions and get an electronic response.
All that messaging can be a burden on doctors and staff, who studies show are already burned out from the yearslong pandemic. There also are shortages of all kinds of health care workers.
Enter the chatbot.
In the first-of-its-kind study, researchers in Baltimore and San Diego turned to Reddit’s AskDocs, a social media forum where patients publicly post medical questions and verified human physicians respond.
Researchers collected 195 of the exchanges, taking out all the patient identifying information. They provided the AskDocs answers and a separate set of ChatGPT answers to a panel of three health care providers without telling the panel where the answers originated.
The panel preferred the ChatGPT responses to the doctor responses almost 80% of the time. The panel members said they were nuanced and accurate and of significantly higher quality.
They even found ChatGPT to be more empathic. For example, the chatbot answer to that toothpick question begins, “It’s natural to be concerned if you have ingested a foreign object, but in this case, it is highly unlikely that the toothpick you swallowed will cause you any serious harm.”
The real doctor wrote, “If you’ve surpassed 2-6 h, chances are they’ve passed into your intestines.”
The doctor went on to write 58 words in total, while the chatbot answer was more than three times as long at 191 words.
John W. Ayers, the study leader and an affiliate scientist at the Qualcomm Institute at the University of California San Diego, said opportunities for artificial intelligence to improve health care “are massive,” and that “AI-augmented care is the future of medicine.”
The researchers say they don’t foresee models set free to answer patients’ questions, at least not yet. A doctor would review the answers prepared by the chatbot, which would have patient records and medical background available to prepare a draft.
The chatbots have shown they can answer medical questions generally, enough to pass medical exams the majority of the time. They can still be wrong, research shows. Also, experts say some people may not trust or be put off by an exchange solely with software. Studies also show another significant red flag.
Harvard University researchers found “algorithmic bias,” which is where the chatbots don’t always properly consider socioeconomic status, race, ethnic background, religion, gender, disability or sexual orientation. That can exacerbate inequities in health systems.
For its part, the American Medical Association, the nation’s largest association of doctors, says artificial intelligence is all but inevitable and can improve the strain on health care providers and the health care system, as well as improve patient care. Its policy on its use will evolve with the technology.
No matter what, chatbots aren’t just on their way, they are here.
Microsoft Corp. and Epic announced a collaboration earlier this month to incorporate artificial intelligence into electronic health record software. Epic is the platform used by many hospitals and doctor offices to house patient records and communicate with patients.
The idea is to increase productivity, enhance patient care and improve the finances of the institutions using the software, according to a news release from the companies.
The AI-powered system is already drafting messages for health providers at UC San Diego Health; UW Health, which is the University of Wisconsin-Madison’s health system; and Stanford Health Care.
“A good use of technology simplifies things related to workforce and workflow,” said Chero Goswami, chief information officer at UW Health, in a release. “Integrating generative AI into some of our daily workflows will increase productivity for many of our providers, allowing them to focus on the clinical duties that truly require their attention.”
Hopkins’ Dredze said he plans to continue studying the potential promise, and pitfalls, of chatbots in health care.
“The focus should not be on replacing doctors,” he said. “It should be on helping doctors do a better job.”
This story has been updated to reflect that UW Health, the University of Wisconsin-Madison’s health system, is one of the institutions using the AI-powered system.
