Bioinspired high-order in-sensor spatiotemporal enhancement in van der Waals optoelectronic neuromorphic electronics

In over-complicated machine vision, target tracking within deep learning paradigms yields inaccurate and energy-intensive outputs. Although spiking neural networks excel at processing dynamic information, challenging tracking environments demand further enhancement in feature correlation learning for efficient target tracking. Distinct from Paired-spike-timing-dependent-plasticity-based architectures, we demonstrate a visual sensor based on van der Waals phototransistors, leveraging Triplet-spike-timing-dependent plasticity to extract bioinspired high-order correlation information, through tunable light-electric cooperation and competition effect on synaptic plasticity originating from interfacial defects-dominated persistent photoconductance phenomena. The universal Triplet-spike-timing-dependent plasticity with enhanced spatiotemporal correlation learning characteristic renders spiking neural networks with better processing capabilities for confusing object classification and dynamic tracking (90.44%) tasks, excelling particularly in seamless tracking post-occlusion, furthermore experimentally validated through hardware implementation on a 6 $$\times$$ 6 van der Waals phototransistor array. The offers a bottom-up methodology employing device physics to guide mapping of biorational learning for high-performance dynamic tracking towards advanced machine visual technologies. Li et al. report a high-order spatiotemporal processing vision sensor based on vdW phototransistors, leveraging Triplet-spike-timing dependent plasticity to extract the high-order correlation information, rendering spiking neural network for confusing object classifying and dynamic tracking.