Machine Learning Algorithms Accurately Predict Risk Factors for Failure to Rescue After Coronary Artery Bypass Grafting

R. Manyam¹, H. Shen², Z. Liu², Y. Zhang³, X. Hu⁴, W. Brent. Keeling^1
1Emory University, Atlanta, Georgia ²Rollins School of Public Health, Emory University, Atlanta, Georgia ³Georgia State University, Atlanta, Georgia ⁴Nell Hodgson School of Nursing, Emory University, Atlanta, Georgia

Purpose: Failure to rescue (FTR) is a significant quality measure of care. Current FTR predictive models use statistical methods that assume linear associations among patient’s clinicopathological characteristics. We developed a machine learning (ML) model to handle nonlinear relationships and predict risk factors for FTR after coronary artery bypass grafting (CABG) accurately.

Methods: Adults who underwent isolated CABG at a multi-hospital academic health system from 2010 to 2022 were queried from an institutional adult cardiac surgical database. The primary outcome was FTR, defined as a 30-day postoperative mortality after a stroke, renal failure, reoperation, or prolonged ventilation. FTR was derived for each of the above postoperative complications. We evaluated 49 potential pre-, intra- and postoperative variables using eXtreme Gradient Boosting (XGBoost), Recursive Feature Elimination (RFE) with cross-validation (CV) ML techniques to determine the optimal subset of covariates with the highest predictive value. Five ML algorithms [Table 1] were constructed on the training data using 10-fold CV and validated on the test data of patients. The relative importance of the risk factors, area under the receiver operating characteristics (AUROC), calibration and Brier score were used to assess and quantify the performance of ML algorithms.

Results: A total of 9974 patients were identified, and the overall observed FTR rate after any complication was 2.5% (n=249). FTR rates for renal failure were 25.2%, 10.4% for stroke, 12.5% for reoperation, and 11.4% for prolonged ventilation. Using the RFE feature selection algorithm and 10-fold CV, the XGBoost ML model demonstrated a cross-validated F1-score of 0.972 (i.e., a harmonic mean of the precision and recall) when the number of variables selected were 14 [Figure 1A]. Of these 14 variables, twelve covariates resulted with importance scores of greater than 50. The twelve risk factors were: age, alcohol use, chronic lung disease, hematocrit, hypertension, immunocompromised present, intraoperative blood products, MELD score, peripheral artery disease, pneumonia, syncope, tobacco use [Figure 1B]. Of the five ML algorithms tested, the XGBoost demonstrated the best performance with an AUC= 0.81, calibration intercept = -0.03, calibration slope: 0.82, and Brier score: 0.02 [Figure 1C and Table 1].

Conclusion: ML algorithms can address the issues of multiple and correlated predictors, non-linear associations and interactions between the predictors and outcome, and process massive amounts of EMR data faster than the traditional methods. The XGBoost algorithm demonstrated good discrimination in identifying patients at high-risk of FTR. The proposed framework can serve as a proof-of-concept for creating intuitive and user-friendly dashboards to provide real time clinical decision support to at-risk patients. Further research on larger patient population, more granular data and external validation is necessary to confirm the prediction ability and clinical utility with respect to FTR risk-stratification of patients undergoing CABG.

Identify the source of the funding for this research project: Department of Biostatistics and Bioinformatics, and Division of Cardiothoracic Surgery in the Department of Surgery, Emory University.