Measuring the Training Effectiveness of Mixed-Reality ECMO Simulations

Introduction: Extracorporeal membrane oxygenation (ECMO) is a high-acuity, low-opportunity life-sustaining technology. The median annual volume of extracorporeal membrane oxygenation (ECMO) cases in American neonatal and pediatric centers is only 9, underscoring the need for supplemental training outside clinical care. While high-fidelity ECMO simulation is considered the gold standard, its adoption is limited by cost and time constraints. Alternative modalities must be explored. This study aims to evaluate the effectiveness of novel mixed reality (MR) ECMO simulations in improving clinician confidence and knowledge in pediatric ECMO management.


Methods: A single-center, pre-post pilot study was performed with a CPR to ECMO (ECPR) MR ECMO simulation. The intended target audience is pediatric critical care medicine fellows, critical care nurse practitioners, and critical care attendings at a tertiary pediatric hospital. Alpha testers of the MR simulation were healthcare simulation educators. Pre-post measures of the primary outcome, confidence rating on ECMO circuit problems and patient scenarios were measured. Secondary outcome domains included training effectiveness (TEI), system usability (SUS), cognitive workload, knowledge, and simulator sickness (SSQ). Qualitative feedback surveys were framed by the Kirkpatrick Training Evaluation Model (KTEM).


Results: Results from alpha testing of the ECPR simulation with 4 healthcare simulation educators included the TEI survey, SUS, KTEM questions, and SSQ. The mean TEI score was 4.8/5. The mean SUS score was 91.3 with an average overall rating of “excellent”. Responses to KTEM questions described the simulation as immersive, well-paced, and educationally valuable, emphasizing strong cognitive engagement. Growth opportunities included allowing for decision-making consequences and better approximating realistic stress levels of ECPR in real-world clinical settings. The average SSQ score was 1.5.


Conclusions: ECPR mixed reality simulation scores highly in training effectiveness and usability among healthcare simulation educators. However, opportunities for improvement include optimizing stress inoculation, enhancing learning through meaningful consequences for incorrect decisions, and implementing additional strategies to reduce simulator sickness symptoms.