A Hybrid Deep Learning Model for Spotted Buffalo Image Detection and Segmentation Using YOLO, VGGNet, and Canny Edge Detection

This study addresses the research gap in the visual recognition of Toraja buffalo, particularly the two main classes, Bonga and Saleko, which exhibit complex morphological variations and diverse coat patterns. Conventional methods often suffer from misclassification under extreme lighting conditions or when the background closely resembles the object. To bridge this gap, a hybrid model is proposed by integrating You Only Look Once (YOLO) for object detection, Visual Geometry Group 16 (VGG16) for deep feature extraction, and Canny edge detection for precise segmentation. The model was evaluated on a dataset of 200 Toraja buffalo images, divided into 70% training, 15% validation, and 15% testing. Experimental results indicate competitive detection performance with mean Average Precision (mAP)@0.5 of 0.84, a Dice coefficient of 0.933, and classification accuracy close to 95% on the confusion matrix. The reliability diagram further confirms that the model's confidence scores are well calibrated, with an Expected Calibration Error (ECE) of 0.03, indicating that the model's predictions align well with empirical accuracy across various confidence levels. The novelty of this research lies in the hybrid approach that combines detection, feature extraction, and edge-based segmentation into a unified framework tailored for Toraja buffalo recognition, a domain rarely explored in previous works. This study contributes not only to the cultural conservation and documentation of Toraja traditions but also offers practical implications for livestock identification systems in complex real-world environments.