ChatGPT and Doctor Google (AI part 3)

September 26, 2023

As long as there has been an internet, people have asked the internet their medical questions.  And for just as long, doctors have been unhappy when their patients bring the answers from “Dr Google” to their offices. Plain web searches often highlight promoted sites selling products that are often not evidence based, and many websites are sponsored by providers or pharmaceutical companies and encourage seeking medical care when a bit of patience would have been the better course of treatment.

Large language models like ChatGPT, though, offer answers that are often far better than a simple web search. Researchers have published multiple articles recently showing that large language models:

●      Are as accurate as an ophthalmologist at addressing general eye questions

●      Show more empathy than physicians on Reddit

●      Offer better answers over time with additional experience and training

●      Can generate effective case studies to teach clinicians 

●      Can judge surgical skills of laparoscopic surgeons

●      Provide good answers to cancer questions, though these were written at a college grade level which many patients would not understand.

Implications for employers:

-        We are certainly not yet at a point where we can rely on generative AI models to provide medical care.

-        However, AI models are already outperforming web searches, and they will continue to improve over time.

-        Employers should be sure that they check their own and vendor AI output for accuracy, cultural sensitivity, and readability.

Tomorrow: Mitigating risks of AI in heath care

Thanks for reading.  You can find previous posts in the Employer Coverage archive

Please “like” and suggest this newsletter to friends and colleagues. Thanks!

Illustration by Dall-E with prompt “impressionist painting of a female physician on a computer screen” Note that with a prompt of “physician” Dall-E only presented male physicians, and I couldn’t get AI to show me what I really wanted, which was the image of a physician actually on the screen!”