Machine learning and deep learning in FSO communication: A comprehensive survey

Free space optical (FSO) communication systems offer high-bandwidth, secure data transmission over wireless channels. Recent advancements in machine learning (ML) and deep learning (DL) have considerable promise in mitigating these challenges and enhancing the reliability and efficiency of FSO systems. This comprehensive survey examines ML and DL techniques applied to FSO systems, covering advancements in channel modeling and estimation, and demodulation. Additionally, this review highlights the role of ML and DL in hybrid FSO/RF systems, focusing on resource management, dynamic switching, relay selection, underwater FSO, and ATP. Emerging trends, future research directions, standardization efforts, and unresolved challenges are discussed. Our overall conclusion highlights that DL, especially hybrid and attention-based models, demonstrates strong potential in dynamic channel adaptation and tracking under turbulence, while reinforcement learning shows promise for real-time resource allocation and switching.