Comparing AI's Large Language Models to Physicians: Clinical Reasoning and Humanism in Critical Care

Introduction: Artificial Intelligence (AI)'s large language models (LLMs) like ChatGPT, Gemini, and Claude are increasingly utilized in clinical settings, and questions emerge regarding their capacity to simulate human reasoning. This study assesses the performance of LLMs across diagnostic, pharmacologic, and ethical tasks, emphasizing their inherent limitations in replacing physicians - particularly in areas such as empathy, judgment, and contextual adaptability.


Methods: Twenty standardized clinical scenarios (Details in full manuscript) spanning diagnosis, management, pharmacology, communication, and patient safety were presented to five different LLMs: ChatGPT (OpenAI), Claude (Anthropic), Gemini (Google), Grok (xAI), and LLaMA (Meta). Responses were evaluated for "gaps, differences, and similarity" in diagnostic reasoning, mechanistic depth, clinical decision-making, empathy, and communication.


Results: Gemini and Grok produced the most comprehensive differentials, with pathophysiologic rationale and rare diagnoses. ChatGPT and Claude offered concise, exam-relevant answers; LLaMA lacked nuance. For acute management, Grok was strongest in emergency care planning, while Gemini provided in-depth but occasionally excessive detail. Claude and ChatGPT were structured yet less robust in contingency planning; LLaMA underperformed. Gemini excelled in mechanistic teaching; Grok blended didactics with pragmatism. ChatGPT and Claude summarized effectively but lacked context. In empathy tasks, Claude and Gemini best utilized communication frameworks (e.g., SPIKES); Grok was procedural; ChatGPT was templated; LLaMA was simplistic.


Conclusions: While LLMs demonstrate impressive capabilities in various areas, they lack the clinical intuition that humans possess—the nuanced, context-driven reasoning applied when facing uncertainty. AI is yet to perceive subtle cues, adjust priorities in real time, or recognize when something feels "off." Our findings indicate that LLMs can assist clinicians, but cannot replace them. Effective clinical care demands more than just pattern recognition. The most promising approach to improving patient care lies in the collaborative use of AI alongside physicians. Probably the future of Critical Care practice lies in the horizon of Human-AI Teaming (HAIT).