Leveraging Data Science for Conducting Observational Studies: Highlighting Advantages and Limitations Throughout the Evaluation of a Use Case.

OBJECTIVE: To develop a data science pipeline for data extraction and collection to conduct an observational study using institutional medical informatics and artificial intelligence tools. PATIENTS AND METHODS: Skin excision cases exposed to intra-incisional clindamycin (from February 5 to October 14, 2025) and nonexposed (from January 1 to December 31, 2022) were defined as our cohorts of interest. The outcome of our study was surgical site infection (SSI) 30 days after surgery. A large language model (LLM)-supported data science pipeline was used for cohort identification, case screening, and extraction of procedural information. LLM screening and data extraction were validated in 300 random cases. A generalized estimating equation model was used to analyze the effect of intra-incisional clindamycin on SSI. RESULTS: The LLM achieved high accuracy for screening cases, with accurate identification of procedures done in the head (accuracy, 99.0%; 95% CI, 97.1%-99.8%) and skin excisions (97.0%; 95% CI, 94.4%-98.6%). In addition, it achieved remarkable performance for extraction of clinical information, with the highest accuracy observed for extraction of anatomical location associated with the procedure (accuracy, 100.0%; 95% CI, 98.8%-100.0%). The final dataset included 2247 skin excision cases (594 exposed and 1653 nonexposed) from 1923 patients. Cases that received intra-incisional clindamycin had lower odds of SSI; however, this association was not statistically significant (aOR, 0.57; 95% CI, 0.31-1.07). CONCLUSION: The implementation of medical informatics tools and robust artificial intelligence algorithms will enable the conduct of end-to-end research pipelines. However, it is important to always consider humans in the loop, as oversight ensures reliability, accuracy, and robustness.