Pediatric Risk of Illness Mortality Evaluation (PRIME) Performance at Quaternary Children’s Hospital

Tuesday, March 24, 2026

8:55 AM - 9:55 AM Central Time

Location: Connections Central Theater 1

SN

Sarah K. Nutman, MD (she/her/hers)

Children's Hospital of Pittsburgh of UPMC

Disclosure(s): No relevant financial relationship(s) to disclose.
JK

Jesse Klug, BBA, MS

Clinical Data Analyst
UPMC

Disclosure information not submitted.
HH

Harry Hochheiser, PhD

Associate Professor
University of Pittsburgh Department of Biomedical Informatics

Disclosure information not submitted.
CB

Chelsea Bitler, MS, DO

Neonatal-Perinatal Medicine & Clinical Informatics Fellow
UPMC Children's Hospital of Pittsburgh

Disclosure information not submitted.
JR

Jesse Raffa, PhD

Research Scientist
Massachusetts Institute of Technology

Disclosure information not submitted.
TP

Tom Pollard, PhD

Research Scientist
Massachusetts Institute of Technology

Disclosure information not submitted.
LC

Leo Anthony Celi, MD, MPH, MS

MD, MPH, MS
Beth Israel Deaconess Medical Center BIDMC

Disclosure information not submitted.
RC

Robert Clark, MD

Professor of Pediatric Critical Care Medicine, Endowed Chair of Pediatric Critical Care Medicine
Department of Critical Care Medicine, Safar Center for Resuscitation Research, John G. Rangos Research Center, UPMC Children’s Hospital of Pittsburgh, University of Pittsburgh School of Medicine, Pittsburgh, PA 15224, USA

Disclosure information not submitted.
CH

Christopher Horvat, MD, MHA

University of Pittsburgh, Pennsylvania

Disclosure information not submitted.

Introduction: There is an ongoing need for generalizable risk adjustment models within intensive care and a growing need to update pediatric models, which were developed using data from more than 10 years ago. Curated real world data sources, such as the electronic health record (EHR), provide the ability to incorporate more information and automate calculation. We developed the Pediatric RIsk of Mortality Evaluation (PRIME) and evaluated its ability to predict mortality in critically ill children.

Methods: This is a retrospective cohort study of admissions to a quaternary PICU from Oct. 2015-Dec. 2021. EHR data were curated from the first 24 hours of PICU admission. Patients were assigned randomly to training (70%) or test (30%) datasets. A ridge logistic regression model was trained on a hospital mortality outcome, with tuning performed using 10-fold cross-validation. Extreme gradient boosted and random forest models were trained for comparison. Model performance was assessed by examining area under the receiver operating curve (AUROC), area under the precision recall curve (AUPRC), standardized mortality ratios (SMR), calibration plots, Brier scores, and the Hosmer-Lemeshow test; performance was compared using the DeLong test. Data are presented with counts (%), medians (interquartile range), and 95% confidence intervals.

Results: There were 15241 encounters with 286 (1.9%) deaths. There were 6738 (44.2%) females and median age was 6 (1-13), with no significant differences between the training and test datasets. There were 7964 (52.2%) admissions from the ED, 2786 (18.2%) from other hospitals, 2260 (14.8%) from acute care, 1918 (12.5%) post-operatively, and 313 (2.1%) from other ICUs. Primary admission reasons were respiratory (5365, 35.2%) and injury/toxic ingestion (2093, 13.8%). PRIME demonstrated excellent discrimination with an AUROC of 0.93 [0.91-0.96] and an AUPRC of 0.46 [0.35-0.57]. Visualized calibration was good, the Hosmer-Lemeshow statistic had P=0.47, the SMR was 1.0, and Brier score was 0.013. PRIME significantly outperformed the electronic Global Open Source Severity of Illness Score (eGOSSIS; P=0.01).

Conclusions: PRIME is a modern, high-performing mortality prediction algorithm. It will be refined using a multicenter international cohort and validated on holdout data from after 2021.