Zero-Shot Detection of Baleen Whale Calls with Vision Foundation Model and Prompt Engineering

Abstract Accurate detection and temporal segmentation of baleen whale calls are essential for understanding behavioral ecology and supporting marine conservation. Land-based seismic stations provide a cost-effective platform for passive acoustic monitoring; yet, strong background noise and limited labeled data restrict the performance of conventional methods. This study introduces a zero-shot detection framework that applies the segment anything model (SAM) to automatically identify and segment Bryde’s whale calls recorded at the Xieyang Island seismic station in the Beibu Gulf, South China Sea. A multistage denoising workflow combining spectral subtraction, Gaussian smoothing, and slice-adaptive thresholding was developed to enhance signal-to-noise ratio (SNR). Guided by prompt engineering and five bioacoustic constraints, SAM performs semantic segmentation on spectrograms, delineating individual call boundaries instead of only classifying presence or absence. Quantitative comparisons against two representative baselines further validated the framework’s superiority, with SAM achieving over 96% precision and recall across both seasonal datasets while avoiding the false positives and missed low-amplitude calls that plagued time-domain methods. Cross-regional validation on fin whale recordings from Ireland and blue whale recordings from Canada demonstrated that the preprocessing workflow and SAM framework generalize across species and marine regions with only parameter adaptation, confirming their broader applicability. Analysis of inter-pulse interval (IPI) in Bryde’s whale calls revealed clear seasonal variation, with shorter IPIs in winter (mean 6.93 s) and longer IPIs in summer (mean 11.70 s). These findings highlight the potential of foundation models, when guided by domain knowledge, to enable label-efficient and scalable bioacoustic monitoring of marine mammals.