Skip to main content

ChatGPT, Large Language Models and NLP – an Informatics Perspective in Healthcare

healthcare professionals using computer chatGPT

My previous blog talked about the opportunities and downsides to ChatGPT for clinicians. Accuracy, bias and privacy were key considerations identified when using ChatGPT. With that being said, do I think ChatGPT replaces other forms of NLP? No or at least, certainly not any time soon. 

In this blog, I approach ChatGPT and Large Language Models (LLMs) from a more pragmatic informatics perspective. 

The effective use of Natural Language Processing (NLP) in healthcare relies upon three key principles – accuracy, transparency and reproducibility. I cannot think of any applications of NLP in the healthcare domain where all three of these are not key considerations. Overlaying all of these, is the very important element of cost of both compute and human resources. 

Key considerations for deploying AI in healthcare

Accuracy and transparency
ChatGPT has been shown to be very accurate in some early use cases in healthcare. However, its precision and recall are not superior to other forms of NLP, such as other smaller language models (a variety of BERT models exist in healthcare) and rules and ontology-based approaches. Whilst these techniques are not as shiny and new – they are more affordable, and in many cases, more accurate. Furthermore, as ChatGPT is a completely black box – how its answers are generated is not shown to the user, and therefore there is no transparency. In so many applications of NLP in healthcare, transparency is paramount. NLP algorithms that code disease from medical records need to present the evidence for “why” to end users who use their expertise to ensure accurate coding takes place. This significance of human review is relevant to several contexts – appropriate reimbursement, better population health, improved care management and reduced audit risk. 

Reproducibility
When using NLP in healthcare – it is very important that if the input is unchanged – that the answer remains the same. This ensures that other clinicians and scientists can reliably repeat and demonstrate rigor in results. When we look at taking applications of NLP into clinical practice, it is essential that results can be reliably repeated across settings, to foster clinician acceptance and trust of the methods. At IQVIA – our open NLP pipeline ensures that results are consistent and repeatable. Furthermore – it enables expert users to use different components – including LLMs in the pipeline to combine multiple NLP techniques for the most effective solution.  

Cost of processing 1 million medical records
Finally, there’s the cost – processing medical text with ChatGPT would require a huge investment. I asked ChatGPT for the cost to process 1 million medical records – and it provided the following*: 

In this case, processing one million medical records would require approximately 500 million tokens. At the current pricing, processing 500 million tokens would cost between $2 million to $4 million, depending on the volume of usage. However, this is just an estimate, and the actual cost may vary based on several factors, such as the complexity of the medical records, the quality of the data, and the specific requirements of the application.”

Those are eye watering costs – and likely underestimate the cost in real terms as server and hosting costs are not included. And, in most healthcare systems, 1 million records are just a fraction of the unstructured data they possess. Therefore, ChatGPT can only be used in cases where it would make economic sense over the alternative. 

Will ChatGPT & LLMs revolutionize Healthcare?

Possibly but not overnight and probably not in its current form. NLP is a fluid and ever-evolving field of informatics. What is termed “state of the art” uses different techniques in different use cases. Right now, given we’ve been in the space for over 20 years we have a great understanding of the tools that can solve our customers challenges. ChatGPT and large language models offer a potential new tool in the NLP toolkit, but its use should be limited to where it adds unique value over other more appropriate, economical and accurate methods. We are already experimenting with GPT to see where it may add to our existing pipelines – exactly where we will see its unique value is yet to be seen, but I am excited about the potential. 

If you would like to learn more about how IQVIA is already delivering on the “new” potential of ChatGPT – please get in touch.

To learn more about NLP in healthcare, check out our recent webinar.

 

*based on prompt from March 3, 2023

Ready to get started?

Request a Demo

Questions? Ask our experts