Advancing Predictive Analytics in Critical Care: The Role of Tabular Foundation Models

Introduction: Artificial Intelligence models for predictive analytics have been extensively researched and trialed in critical care. The Tabular Prior-data Fitted Network (TabPFN) is a tabular foundation model that has been proposed to significantly outperform previous methods on datasets with up to 10,000 samples by a wide margin, while requiring substantially less training time. As a generative transformer-based foundation model, this model also supports fine-tuning, data generation, density estimation, and learning reusable embeddings.


Methods: We curated a balanced subset of 6,814 patients from the eICU Collaborative Dataset (N = 139,367), comprising 3,407 survivors and 3,407 non-survivors within 48 hours of ICU admission. Patients with mortality within 4 hours of ICU admission, ICU readmissions, and those with missing data were excluded. The data variables used for modeling included mean arterial pressure, heart rate, Glasgow Coma Scale, presence of mechanical ventilation, oxygen saturation at the time of ICU admission, and ICU mortality status. After splitting the data into training (80%) and testing (20%), we trained TabPFN, Support Vector Machine (SVM), and Random Forest (RF) models to predict patient mortality within 48 hours of ICU admission. We conducted a comprehensive evaluation of TabPFN across predictive performance, data generation, and embedding quality.


Results: TabPFN demonstrated comparable classification performance (ROC AUC: 0.95, F1 score: 0.87) while achieving the fastest training time (0.34s), compared to Random Forest (ROC AUC: 0.93, F1 score: 0.87, training time: 0.59s) and SVM (ROC AUC: 0.93, F1 score: 0.81, training time: 3.14s).TabPFN’s data generation capabilities were validated via the Kolmogorov–Smirnov (KS) test, showing that most synthetic feature distributions closely matched real data (p > 0.01). We observed a performance drop (ROC AUC from 0.92 to 0.67) when using generated embeddings.


Conclusions: In our experimentation, we find TabPFN is a promising foundation model with superior model performance. Coupled with additional capabilities of synthetic data generation, embeddings generation, and fine tuning abilities, TabPFN is a promising model for use in predictive analytics for critically ill patients.