Tackling Challenges in Development and Validation of Predictive Models for Class Imbalanced Data

Music City Center 

Small bowel obstruction (SBO) is common in ED, with operative management recommended within 24hrs (OM) for suspected ischemia or treatment failure, though 80% of cases improve with nonoperative care (NOC). We developed logistic regression (LR) and random forest (RF) models to predict NOC in patients admitted to ED. This secondary analysis uses data from a multicenter retrospective study of SBO patients diagnosed by CT at 10 EDs. 70% of data was used for training, 30% for testing, with stratification to maintain the case-control ratio, and both RF and LR models were weighted for class imbalance. We selected physical exam features, WBC count, creatinine, lactic acid, history of malignancy, and hernia from clinically/statistically significant features using stepwise regression. Of 1419 patients with history of SBO confirmed by CT imaging, 6%patients required OM. The AUROC for LR was 0.68 (95% CI: 0.56-0.79) vs. 0.56 (95% CI: 0.45-0.67) for RF, with no significant difference (P=0.176). The misclassification rate was 42% for LR vs. 43% for RF. Statistical models can help triage SBO patients in the ED, though misclassification persists; cutoff values for 95% sensitivity are provided.

Logistic Regression

Random Forest

Decision Trees

Predictive Models

Small Bowel Obstruction

Non-Operative Management 

Section on Statistical Learning and Data Science