Enhancing Clinical Research Classification with Advanced Machine Learning Models and Hierarchical Approaches

: Background and Objective Our previous work demonstrated that a Random Forest model could successfully automate the 3-class classification of clinical research activities, significantly improving efficiency. Building on this success, this project aimed to enhance classification granularity and accuracy by expanding the dataset, refining the classification schema to four categories, and evaluating a new range of state-of-the-art Machine Learning (ML) models, including Large Language Models (LLMs) and Convolutional Neural Networks (CNNs). Methods The original dataset was expanded to approximately 1230 clinical studies. The classification schema was refined to distinguish between Interventional Studies (IS), Primary Data Collection Non-Interventional Studies (PDC NIS), Secondary Data Use Non-Interventional Studies (SDU NIS) and Real-World Evidence Scientific Projects (RWE SP). We conducted an extensive evaluation of 17 models, including ClinicalBioBERT, RoBERTa and ResNet, against our Random Forest baseline. Both a direct 4-class classification and a 2-step hierarchical approach were tested. Advanced techniques for overfitting mitigation, including early stopping, class-weighted losses, regularisation and Bayesian hyperparameter optimisation, were systematically implemented. Results While advanced LLMs demonstrated strong initial performance, their advantage diminished after rigorous regularisation, a 2-step hierarchical Random Forest model emerged as the superior solution after rigorous testing and optimisation. It achieved an overall accuracy of 95.12% and an F1-score of 95.03%. This approach outperformed both direct 4-class models and the more computationally intensive LLMs, offering an optimal balance of performance, stability and a 41% reduction in execution time. The final model was successfully integrated into a classification application. Conclusion This research confirms that a more granular and robust classification of clinical research can be achieved through ML classification techniques. The success of the hierarchical Random Forest model provides compelling evidence that for this specific, domain-constrained task, a tailored, simpler architecture offers a superior balance of accuracy, speed and interpretability over complex deep learning models accelerating regulatory compliance in the biopharmaceutical industry.