Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation

Published

June 26, 2023

Author(s)

Adam McCaughan, Bakhrom Oripov, Natesh Ganesh, Sae Woo Nam, Andrew Dienstfrey, Sonia Buckley

Abstract

We show that model-free perturbative methods can be used to efficiently train modern neural network architectures in a way that can be directly applied to emerging neuromorphic hardware. These methods were investigated for training VLSI neural networks beginning in the 1990s, and more recently on memristive crossbars and photonic hardware, but all these demonstrations have been very limited in scale, comprising small datasets with only a few neurons. We describe a framework for applying these techniques to existing neuromorphic hardware at much larger scales, with an emphasis on creating simple, highly-localized circuits that could be implemented on-chip if desired. The framework is also extensible to training existing hardware systems via a chip-in-the-loop technique.

Citation

APL Machine Learning

Pub Type

Journals

Download Paper

https://doi.org/10.1063/5.0157645

Local Download

Keywords

machine learning, neuromorphic

Citation

McCaughan, A. , Oripov, B. , Ganesh, N. , Nam, S. , Dienstfrey, A. and Buckley, S. (2023), Multiplexed gradient descent: Fast online training of modern datasets on hardware neural networks without backpropagation, APL Machine Learning, [online], https://doi.org/10.1063/5.0157645, https://tsapps.nist.gov/publication/get_pdf.cfm?pub_id=936062 (Accessed April 28, 2024)