Researchers at Tohoku University and the University of California, Santa Barbara have shown a proof-of-concept of energy-efficient computer compatible with current AI. It utilizes a stochastic behavior of nanoscale spintronics devices and is particularly suitable for probabilistic computation problems such as inference and sampling.
They presented the results at the IEEE International Electron Devices Meeting (IEDM 2023) on December 12, 2023.
With the slowing down of Moore’s Law, there has been an increasing demand for domain-specific hardware. Probabilistic computer with naturally stochastic building blocks (probabilistic bits, or p-bits) is a representative example owing to its potential capability to efficiently address various computationally hard tasks in machine learning (ML) and artificial intelligence (AI). Just as quantum computers are a natural fit for inherently quantum problems, room-temperature probabilistic computers are suitable for intrinsically probabilistic algorithms, which are widely used for training machines and computational hard problems in optimization, sampling, etc.
Recently, researchers from Tohoku University and the University of California Santa Barbara have shown that robust and fully asynchronous (clockless) probabilistic computers can be efficiently realized at scale using a probabilistic spintronic device called stochastic magnetic tunnel junction (sMTJ) interfaced with powerful Field Programmable Gate Arrays (FPGA).
Until now, however, sMTJ-based probabilistic computers have been only capable of implementing recurrent neural network, and developing the scheme to implement feedforward neural networks have been awaited. “As the feedforward neural networks underpin most modern AI applications, augmenting probabilistic computers toward this direction should be a pivotal step to hit the market and enhance the computational capabilities of AI” said Professor Kerem Camsari, the Principal Investigator at the University of California, Santa Barbara.
In the recent breakthrough to be presented at the IEDM 2023, the researchers have made two important state-of-the-art advances. First, leveraging earlier works by the Tohoku University team on stochastic magnetic tunnel junctions at the device level, they have demonstrated the fastest p-bits at the circuit level by using in-plane sMTJs, fluctuating every ~microsecond or so, about three orders of magnitude faster than the previous reports. Second, by enforcing an update order at the computing hardware level and leveraging layer-by-layer parallelism, they have demonstrated the basic operation of the Bayesian network as an example of feedforward stochastic neural networks.
“Current demonstrations are small-scale, however, these designs can be scaled up by making use of CMOS-compatible Magnetic RAM (MRAM) technology, enabling significant advances in machine learning applications while also unlocking the potential for efficient hardware realization of deep/convolutional neural networks,” said Professor Shunsuke Fukami, the Principal Investigator at Tohoku University.
- Publication Details:
Title: Hardware Demonstration of Feedforward Stochastic Neural Networks with Fast MTJ-based p-bits
Authors: Nihal Sanjay Singh, Shaila Niazi, Shuvro Chowdhury, Kemal Selcuk, Haruna Kaneko, Keito Kobayashi, Shun Kanai, Hideo Ohno, Shunsuke Fukami and Kerem Y. Camsari
Conference: 69th Annual IEEE International Electron Devices Meeting (IEDM 2023)
