TLDR: When using AI models to predict ventilator outcomes, is medication a good predictor? Why does performance always seem to be capped around 0.80? Is this limit due to model choice, input data type, or simply irreducible noise when predicting across diverse ICU populations?

Machine Learning for Mechanical Ventilation

⸂⸂⸜(രᴗര๑)⸝⸃⸃ Hey everyone!! 👋 Biomed engineering PhD student here - Always interested in seeing new ways technology might contribute to the healthcare space. Anyways, this study here recently caught my attention, and I’m curious to hear your thoughts. 

During COVID-19, mechanical ventilation quickly became a hot topic. First thanks to global ventilator shortages and later due to the less than ideal outcomes from machine ventilation use. High mortality and complications like pneumonia raised an uncomfortable question: are we using mechanical ventilation for the right patients, at the right time, for the right duration?

To answer this question, many researchers have jumped onto the AI boat, leveraging  machine learning (ML) to analyze patient data. Several publications have emerged since then, such as:

• Xu et al., 2024 used Random Forest to predict weaning difficulty in ventilated ICU patients, achieving 0.805 AUROC with 36 variables. 

• Agard et al., 2025 : utilised a long short-term memory (LSTM) network. They predicted ventilator-associated pneumonia (VAP) 6 - 24 hours before it became clinically obvious and hit 94 - 96% AUPRC with 5 variables.

A new Pharmacotherapy study, Murray et al., 2026, also made use of the very popular Random Forest and SVM models, but took a different approach in terms of input variables, where medication regimen complexity was used as a predictor of prolonged ventilation (>5 days) in the ICU. 

• The authors developed a “MRC-ICU” score to systematically quantify medication regimen complexity, assigning weighted values to 35 distinct medication categories, generating a total score. 

• Their justification came from the fact: 70% of patients in the ICU are receiving more than 13 medications at any given time. 

• Previously, medication data was rarely highlighted as a primary predictor, as:

  1. Medication data was difficult to compile and integrate. 

  2. Other papers evaluated medication to be non-significant

While the overall accuracy of Murray’s MRC-ICU model (~0.78 AUROC) is not dramatically higher than other existing models, the authors argued that their study differs in scope: it can predict over a broad ICU population rather than a narrowly defined cohort, like traumatic brain injury Prediction of in-hospital mortality in patients on mechanical ventilation post traumatic brain injury: machine learning approach or congestive heart failure Machine Learning-Based Model for Predicting Prolonged Mechanical Ventilation in Patients with Congestive Heart Failure. They suggest that this model: may be best suited for identifying patients unlikely to require prolonged mechanical ventilation as opposed to confidently identifying patients at high risk.

👉 Performance gap: Model choice or Input Variable?

Thus far, many studies (including Xu et al., 2024 and Murray et al., 2026) using the traditional tree-based methods seem to hit a ~0.78-0.80 AUROC ceiling, while Agard's LSTM broke through to over 94% precision with just 5 variables. 

At face value, the advantage appears to be temporal awareness. Agard’s model was fed with 24-hour sequences of vital signs, capturing trends, volatility, and timing. But, it also brings forth the question:

Is this performance ceiling a limitation of tree-based architectures or simply a reflection of richer, more granular input data? 

At the same time, given the ‘just ok’ performance (~0.78 AUROC) for Murray’s model, I think there are, in fact, more underlying uncertainties:

1. Is medication complexity genuinely useful and pulling its weight as a predictor?

2. Is MRC-ICU just a proxy for disease severity, and is it a good one when predicting across diverse ICU populations?

3. Are Random Forests becoming obsolete with all the new architecture available? Is it time to leave it behind?

(∿°○°)∿ Drop your thoughts!  What do you all think?