Recently, OpenAI’s large language model ChatGPT (Chat Generative Pre-trained Transformer) made headlines as it successfully passed a series of United States medical exams. This is a major milestone in the field of Artificial Intelligence (AI) and demonstrates the potential for AI to play a significant role in the medical field.
The medical exams that ChatGPT passed include the United States Medical Licensing Examination (USMLE) Step 1 and Step 2 Clinical Knowledge (CK). The artificial intelligence program scored between 52.4 and 75 percent across the three-part Medical Licensing Exam (USMLE). Each year’s passing threshold is around 60 percent. These exams are widely recognized as the gold standard for medical licensing in the United States, and passing them is a requirement for practicing medicine in the country.
The USMLE exams are designed to test the medical knowledge and clinical skills of aspiring medical professionals. The USMLE Step 1, is usually taken in the medical school at the end of second year. Step 2 is taken in fourth year and finally step 3 is taken after the completion of medical school and the first year of residency. The exams are highly regarded in the medical community and are known for their rigorous content and challenging questions. However, ChatGPT was able to perform well on these exams, demonstrating its mastery of medical knowledge and its ability to understand complex medical concepts.
Scientist from the AnsibleHealth company, who did the research said “Reaching the passing score for this notoriously difficult expert exam, and doing so without any human reinforcement, marks a notable milestone in clinical AI maturation.” and the study has been peer-reviewed and published in the journal PLOS Digital Health.
Comparing the performance of ChatGPT research to that of PubMedGPT, it was evident that OpenAI ChatGPT Out performed the PubMedGPT a counterpart model trained exclusively on biomedical domain literature, which scored 50.8 percent on an older dataset of USMLE-style questions.
One of the key factors that contributed to ChatGPT’s success is its massive training corpus. OpenAI’s language model was trained on a diverse range of data sources, including medical textbooks, journals, and online resources. This allowed ChatGPT to develop a deep understanding of medical concepts and to accurately answer questions related to medical diagnoses, treatments, and procedures. In a statement by Dr Stuart Armstrong, Co-Founder and Chief Researcher at Aligned AI, said ‘this is an impressive performance, and we should expect to see more such successes in AI in the future.
It’s worth noting that ChatGPT’s performance on the USMLE exams is not an indicator that it is capable of practicing medicine. The exams are designed to test knowledge and do not assess a candidate’s ability to perform hands-on procedures or make clinical decisions. However, the results do suggest that AI has the potential to play a significant role in the medical field, especially in areas such as data analysis and information retrieval.
In conclusion, OpenAI’s ChatGPT passing the gold standard US medical exams is a major milestone in the field of AI. The results demonstrate the model’s ability to understand complex medical concepts and accurately answer medical questions. As AI continues to evolve, it is likely that we will see increasing use of AI in the medical field, potentially leading to improved patient outcomes and more efficient healthcare delivery.