Hybrid Event-Frame Sensing for Human-Perceptual Imaging and Machine Vision.

Frame-based RGB image sensors and event-based vision sensors provide complementary sensing capabilities for human-perceptual imaging and machine vision. RGB image sensors capture dense spatial, color, and texture information that is essential for human-viewable imaging, semantic recognition, and conventional image signal processing pipelines. In contrast, dynamic vision sensors (DVSs) and event vision sensors (EVSs) asynchronously detect local brightness changes and provide sparse temporal information with low latency, high temporal resolution, and reduced redundant data output. Because neither modality alone satisfies all requirements of emerging vision systems, hybrid event-frame sensing has become an important direction for compact, low-latency, and energy-efficient sensing. This review presents a sensor-oriented taxonomy of hybrid event-frame sensing architectures and systems, including dual-camera event-frame systems, optically aligned event-frame systems, pixel-level shared hybrid image sensors, stacked CIS-DVS hybrid image sensors, homogeneous-pixel sensing systems, and event-only reconstruction systems. We analyze key sensor specifications, including latency, spatial resolution, color fidelity, power consumption, and form factor, and discuss how these specifications guide sensor configuration and design. The review identifies stacked CIS-DVS sensors as one of the most balanced and competitive architectures because they can support compact integration, synchronized event-frame sensing, and on-chip processing. However, important challenges remain, including color fidelity, demosaicing, event-pixel ratio optimization, calibration, benchmarking, and edge-AI deployment. Finally, we emphasize that future hybrid event-frame sensing systems should be developed through sensor-algorithm-ISP-AI co-design. This review provides practical guidelines for developing next-generation hybrid event-frame sensing systems for both human-perceptual imaging and machine vision.