Large Language Models (LLMs), such as ChatGPT or Bard, hold immense promise but also pose significant challenges for healthcare and medicine. To unlock their enormous benefits, we must ensure their safe application in an environment where lives are at stake. In other words, our task is to establish a robust, ethical framework for these generative AI models – without making the boundaries so tight that it kills innovation.
Our latest paper published in Nature’s npj Digital Medicine, “The Imperative for Regulatory Oversight of Large Language Models (or Generative AI) in Healthcare” published with Dr Eric Topol analysed the challenges and the possibilities of LLM regulations to make them accessible for healthcare use.
We need brand new kinds of regulations
Regulators must cover myriad scenarios and perspectives since LLMs significantly differ from existing and regulated AI algorithms and deep learning methods.
A range of distinct characteristics sets them apart, including their:
- scale and complexity – LLMs utilize billions of parameters, resulting in unprecedented complexity. Tokenization, their basic “processing” method is currently not covered by healthcare regulators
- broad applicability – LLMs have unprecedented versatility compared to specialized deep learning models. As they can be used in various sectors, from finance to healthcare, one-size-fits-all regulations will not suffice
- real-time adaptation – LLMs can adapt their responses based on user inputs and the evolving context of the “conversation”, sharing similarities with adaptive algorithms
- data privacy and security – a multifaceted problem-complex, which spans from how to address the ownership issues of the training data – with only a slight exaggeration: the content of the internet and all libraries – to what protects user data that was the input of our prompts and to who owns the results of the human-LLM cooperation
The use of LLMs in medical practices brings unprecedented challenges
Despite their potential, LLMs also amplify risks and challenges in healthcare. To date, no LLM has had pre-training with the corpus of medical information or with millions of patient records, images, lab data, and office visits or bedside conversations – although according to news reports, Google’s MedPaLM could be such an LLM, but it is inaccessible for laymen.
Details about the training process for GPT-4, the most advanced publicly available LLM at the moment, remain undisclosed. Nevertheless, these models have immense potential in medicine, with use cases ranging from clinical documentation to providing personalized health plans.
However, current models can also “hallucinate” results or provide misleading information, which, in medical settings could lead to incorrect diagnoses or treatment recommendations. Biases in training data can affect clinical decision-making, patient outcomes, and healthcare equity. Patients may also come to medical consultations with information from chatbots, complicating the physician-patient dynamic.
We need to regulate the future, not just the present
Typical (publicly available) advanced LLMs work with text inputs at the moment. But we also know that these models can already analyze images, and it can be expected that they will soon deal with uploaded documents, research papers, hand-written notes, sound, and video. This means future regulations have to cover these future use cases
Companies with approved devices that decide to implement LLMs into their services face an additional challenge. Namely, how will the FDA regulate an AI-based medical technology recently infused with LLM if the technology was already approved for medical uses?
Our practical expectations
Here is our summary of what I expect regulators will do in this regard.
- Create a new regulatory category for LLMs
- Provide regulatory guidance for companies and healthcare organizations about how they can deploy LLMs into their existing products and services
- Create a regulatory framework that not only covers text-based interactions but possible future iterations such as analyzing sound or video
- Provide a framework for making a distinction between LLMs specifically trained on medical data and LLMs trained for non-medical purposes.
- Similar to the FDA’s Digital Health Pre-Cert Program, regulate companies developing LLMs instead of regulating every single LLM iteration
By taking a proactive approach to regulation, it is possible to harness the potential, minimize potential harm and preserve the trust of patients and healthcare providers alike. It’s also intriguing to consider that LLMs could potentially become the first category of AI-based medical technologies regulated through the implementation of patient-focused design.
The post Why And How To Regulate ChatGPT-Like Large Language Models In Healthcare? appeared first on The Medical Futurist.
