Researchers at the Massachusetts Institute of Technology have developed a machine learning model to help predict patient mortality in the ICU. The model trains specifically on patient subpopulations, but also shares data across all subpopulations to get better predictions. In doing so, it can better predict a patient’s risk of mortality during their first two days in the ICU, compared to strictly global and other models. “ICUs are very high-bandwidth, with a lot of patients,” says first author Harini Suresh, a graduate student in the Computer Science and Artificial Intelligence Laboratory (CSAIL). “It’s important to figure out well ahead of time which patients are actually at risk and in more need of immediate attention.”
The model first crunches physiological data in electronic health records (EHR) of previously admitted ICU patients, some who had died during their stay. In doing so, it learns high predictors of mortality, such as low heart rate, high blood pressure, and various lab test results over the first few days and breaks the patients into subpopulations based on their health status. Given a new patient, the model can look at that patient’s physiological data from the first 24 hours and, using what it’s learned through analyzing those patient subpopulations, better estimate the likelihood that the new patient will also die in the following 48 hours.
Moreover, the researchers found that evaluating (testing and validating) the model by specific subpopulations also highlights performance disparities of global models in predicting mortality across patient subpopulations. This is important information for developing models that can more accurately work with specific patients.
During training, the model separates patients into distinct subpopulations, which captures aspects of a patient’s overall state of health and mortality risks. It does so by calculating a combination of physiological data, broken down by the hour. Physiological data include, for example, levels of glucose, potassium, and nitrogen, as well as heart rate, and respiratory rate.
Next, the model employs a multitasking method of learning to build predictive models. When the patients are broken into subpopulations, differently tuned models are assigned to each subpopulation. Each variant model can then more accurately make predictions for its personalized group of patients. This approach also allows the model to share data across all subpopulations when it’s making predictions. When given a new patient, it will match the patient’s physiological data to all subpopulations, find the best fit, and then better estimate the mortality risk from there. “We’re using all the patient data and sharing information across populations where it’s relevant,” Suresh says. “In this way, we’re able to … not suffer from data scarcity problems, while taking into account the differences between the different patient subpopulations.”
“Patients admitted to the ICU often differ in why they’re there and what their health status is like. Because of this, they’ll be treated very differently,” says CSAIL graduate student Jen Gongadds. Clinical decision-making aids “should account for the heterogeneity of these patient populations … and make sure there is enough data for accurate predictions.”
A key insight from this method, Gong says, came from using a multitasking approach to also evaluate a model’s performance on specific subpopulations. Global models are often evaluated in overall performance, across entire patient populations. But the researchers’ experiments showed these models actually underperform on subpopulations. The global model tested in the paper predicted mortality fairly accurately overall, but dropped several percentage points in accuracy when tested on individual subpopulations.
Such performance disparities are difficult to measure without evaluating by subpopulations, Gong says: “We want to evaluate how well our model does, not just on a whole cohort of patients, but also when we break it down for each cohort with different medical characteristics. That can help researchers in better predictive model training and evaluation.”
Source: Massachusetts Institute of Technology