Evaluating the Performance of Traditional Pharmacoepidemiologic and Machine Learning Models to Predict Pregnancies at Risk of Major Congenital Malformations.

BACKGROUND: With approximately 50% of pregnancies being unplanned, there is an unintended exposure to potential feto-toxic drugs that may cause major congenital malformations (MCM). This study aims to compare the predictive performance between traditional pharmacoepidemiologic (PE) and machine learning (ML) models. METHODS: We conducted a cohort study within the Quebec Pregnancy Cohort, including all pregnancies covered by Quebec's prescription drug insurance program and their children from 01/1998 to 12/2015. Medication exposures, comorbidities, and women's characteristics 12 months before pregnancy and during the first trimester were considered. Robust Poisson models were used to obtain adjusted risk ratios (aRR) and 95% confidence intervals (CI) of predictors. Logistic regression, robust Poisson, K-Nearest Neighbors, Random Forest, XGBoost, Naïve bayes, Multilayer Perceptron, and Support Vector Machine were developed to predict pregnancies at risk of MCM. Sensitivity, specificity, PPV, NPV, accuracy, ROC-AUCs, PR-AUCs, and F1-score were used to evaluate the performance of predictive models. RESULTS: We analyzed 213,744 pregnancies, finding a 9.7% prevalence of MCM. Logistic regression had the highest discriminative power across models at predicting MCM, with a ROC-AUC of 53.3% and the highest sensitivity (41.5%) and F1-score (46.6%). KNN had the highest specificity (97.9%) but the lowest sensitivity (2.3%). Robust Poisson performed similarly to logistic regression, with the highest accuracy (52.3%). Robust Poisson performed slightly better than logistic regression at classifying organ-specific malformations. All models showed poor overall predictive performance. Results were robust across sensitivity analyses. CONCLUSION: There is insufficient evidence for the superiority of ML over traditional pharmacoepidemiologic modeling in predicting MCM.