ICU mortality prediction using fine-tuned large language models on structured clinical data

Clinical methods for mortality prediction rely on manual calculation of composite risk scores based on multiple clinical bio-markers, which present a significant challenge in an ICU setting. Traditional scoring systems like APACHE IV and Charlson Comorbidity index (CCI) are time-consuming, prone to human error, and often struggle with incomplete data which is a frequent challenge in critical care settings. While previous large language model (LLM) based approaches have shown promising advances, they are heavily dependent on cliniciangenerated notes which may introduce delays in time-critical situations. In this work we aim to show that finetuned large scale models like LLaMA-3.2 can serve as effective clinical decision support tools. We have curated a comprehensive set of clinical variables, derived from key features of the APACHE IV and CCI methods, utilizing data from 2,292 ICU patients in the MIMIC-III dataset. We fine-tuned and evaluated three LLMs based on LLaMA-3.2 (1b and 3b) and MMed-LLaMA-8b for mortality prediction in these patients and compared the results. Our results demonstrate that fine-tuned LLaMA-3.2-3b model significantly outperformed other methods, achieving an AUC of 98.97%, accuracy of 95.86% and sensitivity of 97.38%. This demonstrates substantial improvement over the other baselines, suggesting that the LLaMA-3.2-3b model can serve as a valuable tool for clinical assistance to identify high-risk patients requiring urgent interventions. These findings demonstrate that fine-tuned LLMs can be effectively adapted for structured clinical data processing, offering a viable alternative when domain-specific medical datasets are limited.