Breaking the memory wall: next-generation artificial intelligence hardware

The relentless advancement of artificial intelligence (AI) across sectors such as healthcare, the automotive industry, and social media necessitates the development of more efficient hardware solutions that can implement diverse learning algorithms. This lead article explores the evolution of AI learning algorithms and their computational demands, using autonomous drone navigation as a case study to highlight the limitations of traditional hardware. Traditional hardware, based on the von Neumann architecture, suffers from limited computational efficiency due to the separation of compute units and memory, also known as the “memory wall” problem. To overcome this barrier, this article discusses novel approaches to AI hardware design, focusing on compute-in-memory (CIM) techniques and stochastic hardware. CIM offers a promising solution to the memory wall problem by integrating computing capabilities directly into the memory system. This article details state-of-the-art developments in CIM for different memory types and at various levels of the memory hierarchy to support essential AI compute functions. We also discuss the use of CIM in developing neuromorphic hardware capable of accelerating biologically inspired algorithms, such as spiking neural networks. Furthermore, we highlight how stochastic hardware can exploit the error resilience of AI algorithms to enhance energy efficiency. Encompassing the full stack of AI systems, from learning algorithms to circuit and device-level techniques and architectures, this article provides a comprehensive roadmap for future research and development in AI hardware.