High-throughput optical neuromorphic graphic processing at millions of images per second

Abstract Optical diffractive neural networks (DNNs) offer superb parallelism and scalability for the direct analogue processing of planar information. However, their complete reliance on coherent light interference constrains the integration and computational frequency, as well as demonstrating low diffraction efficiency and robustness. Here, we present an optical graphics processing unit (OGPU) with a vertically integrated architecture, addressing these challenges through the use of an addressable vertical-cavity surface-emitting laser (VCSEL) array. This array functions as a high-speed planar information fan-in device, with each unit exhibiting individually coherent and mutually incoherent (MI) properties. We develop MI-DNNs that leverage the direct operations of spatially incoherent light while preserving the benefits of coherent computing. Therefore, the entire computing system with free-space architecture can be miniaturized to a handheld form factor. The OGPU operates efficiently under ultralow-light conditions (as low as 3.52 aJ/μm 2 per frame) and achieves a record image processing speed of 25 million frames per second. The OGPU has a computational power of 77.3 tera-operations per second (TOPS) and an energy efficiency of 950 TOPS/W. The OPGU achieves competitive image classification accuracy of up to 98.6% and serves as versatile parallel convolutional kernels for image processing tasks, including edge extraction and image denoising.