Automated surrogate-assisted inverse design of polarization-tailored optical transfer functions in nonlocal metasurfaces.

Nonlocal metasurfaces offer a compact platform for analog optical computing, but the inverse design of polarization-resolved optical transfer functions (OTFs) remains challenging because structural parameters are strongly coupled to angle- and polarization-dependent responses. Here, we present a surrogate-assisted inverse-design framework that combines a deep-neural-network forward model with Bayesian optimization for C6-symmetric silicon nanohole metasurfaces. The key advance is a polarization-resolved, multi-objective formulation that independently constrains the p- and s-polarized angular transmission responses, enabling customized relationships between the two polarization channels. The complete post-training online workflow, including Bayesian optimization and final full-wave validation, requires approximately 312 s per device. We demonstrate five image-processing functions at selected wavelengths within 1450-1650 nm: polarization-insensitive and strongly polarization-asymmetric second-order differentiation, polarization-insensitive and strongly polarization-asymmetric Gaussian high-pass filtering, and a hybrid processor implementing different operations in the two polarization channels. The numerically designed devices exhibit an effective numerical aperture of approximately 0.34 and average transmission efficiencies above 0.7, with performance verified by full-wave and Fourier-optics simulations. These results establish a flexible workflow for the customized design of polarization-tailored optical processors.