Multi-Site Validation of Bedside Monitoring-Based Machine Learning Model for Predicting Hemorrhage

Introduction: Continuous bedside monitoring data contains subtle physiological patterns that precede critical events, enabling prediction through machine learning models. We report the retrospective multi-site validation of a model developed to predict hemorrhage up to eight hours in advance. The goal was to validate performance across institutions and patient populations.


Methods: A hemorrhage event was defined as the start of transfusion of >3 units of packed red blood cells within a 24-hour period, with none in the preceding 24 hours. The logistic regression model was trained on waveform data of 3,688 adult ICU admissions from the University of Virginia (UVA).

Retrospective validation was conducted across three sites: (1) UVA (2020–2023), (2) University of Pittsburgh Medical Center (UPMC, 2018–2023), and (3) the waveform-matched subset of the MIMIC-III dataset (Beth Israel Deaconess, 2001–2012). Data from adults admitted to a medical or surgical ICU were collected under IRB-approved protocols with waiver of consent and deidentified. A hemorrhage score was calculated every 15 minutes and the area under the receiver operating characteristic curve (AUC) was determined for detection 8 hours before the hemorrhage event, excluding data within 24 hours after the event. Calibration was assessed by comparing predicted risk to observed event frequency via the absolute percent difference.


Results: In the development cohort, the cross-validated AUC was 0.70 (141 events). Across 17,588 external ICU admissions with 968 events, combined AUC was 0.71. Site-specific AUCs were: 0.73 at UVA (203 events), 0.71 at UPMC (591), and 0.71 in MIMIC-III (174). Performance was consistent across sex and race. In patients over 75 years, AUC was lower (0.663), possibly reflecting altered physiological responses with age. Calibration error was low (median 0.7%, IQR 11.5%).


Conclusions: This study demonstrates that a physiologically derived hemorrhage risk score can support early detection of bleeding in critically ill adults using continuous monitoring data. The model generalizes across institutions, monitor types, and patient populations, providing quantitative risk estimates that complement clinical assessment. Temporal patterns in the risk score reflect rising physiologic derangement prior to transfusion.