FLSea: Underwater Visual–Inertial and Stereovision Forward‐Looking Data Sets

ABSTRACT Visibility underwater is challenging and degrades as the distance between the subject and the camera increases. That is why forward‐looking underwater computer vision tasks are difficult. We have collected underwater forward‐looking stereovision and visual–inertial image sets using two underwater imaging platforms, a stereo camera rig, and an ROV in the Mediterranean and Red Seas. To our knowledge, there are no other public data sets in the underwater environment with this forward‐looking camera‐sensor orientation that have published ground‐truth depth maps as well as pose. These data sets are critical for the development of several underwater applications, including autonomous obstacle avoidance, visual odometry, 3D tracking, Simultaneous Localization and Mapping and depth estimation through deep learning. The stereo data sets contain synchronized stereo images, and the visual–inertial data sets include monocular images and inertial measurement unit (IMU) measurements with millisecond‐level timestamp alignment. All data was collected in dynamic underwater environments with objects of known size. Both sensor configurations allow for scale estimation, with the calibrated baseline in the stereo setup and the IMU in the visual–inertial setup. Ground‐truth depth maps were created offline for both data set types using a commercial photogrammetry software (Agisoft Metashape). The ground truth is validated with multiple known measurements placed throughout the imaged environment. There are four stereo and 12 visual–inertial data sets in total, each containing thousands of images, with a range of different underwater visibility and ambient light conditions, natural and man‐made structures, and dynamic camera motions. The forward‐looking orientation of the camera plus the corresponding ground truth makes these data sets unique and ideal for testing underwater obstacle‐avoidance algorithms and for navigation close to the seafloor in dynamic environments. We show results from an experiment with a monocular depth estimation algorithm to demonstrate the applicability of the data sets. With our data sets, we hope to encourage the advancement of autonomous functionality for underwater vehicles in dynamic and/or shallow‐water environments.