Methodology and development of a machine learning probability calculator: Data heterogeneity limits ability to predict recurrence after arthroscopic Bankart repair

Abstract

Purpose: The aim of this study was to develop and train a machine learning (ML) algorithm to create a clinical decision support tool (i.e., ML‐driven probability calculator) to be used in clinical practice to estimate recurrence rates following an arthroscopic Bankart repair (ABR). Methods: Data from 14 previously published studies were collected. Inclusion criteria were (1) patients treated with ABR without remplissage for traumatic anterior shoulder instability and (2) a minimum of 2 years follow‐ up. Risk factors associated with recurrence were identified using bivariate logistic regression analysis. Subsequently, four ML algorithms were developed and internally validated. The predictive performance was assessed using discrimination, calibration and the Brier score. Results: In total, 5591 patients underwent ABR with a recurrence rate of 15.4% (n = 862). Age <35 years, participation in contact and collision sports, bony Bankart lesions and full‐thickness rotator cuff tears increased the risk of recurrence (all p < 0.05). A single shoulder dislocation (compared to multiple dislocations) lowered the risk of recurrence (p < 0.05). Due to the unavailability of certain variables in some patients, a portion of the patient data had to be excluded before pooling the data set to create the algorithm. A total of 797 patients were included providing information on risk factors associated with recurrence. The discrimination (area under the receiver operating curve) ranged between 0.54 and 0.57 for prediction of recurrence. Conclusion: ML was not able to predict the recurrence following ABR with the current available predictors. Despite a global coordinated effort, the heterogeneity of clinical data limited the predictive capabilities of the algorithm, emphasizing the need for standardized data collection methods in future studies. Level of Evidence: Level IV, retrospective cohort study.