Skip to main content
U.S. flag

An official website of the United States government

Official websites use .gov
A .gov website belongs to an official government organization in the United States.

Secure .gov websites use HTTPS
A lock ( ) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.

Layer-Skipping Connections Improve the Effectiveness of Equilibrium Propagation on Layered Networks

Published

Author(s)

Jimmy I. Gammell, Sae Woo Nam, Adam McCaughan

Abstract

Equilibrium propagation is a learning framework that marks a step forward in the search for a biologically-plausible implementation of deep learning, and could be implemented efficiently in neuromorphic hardware. Previous applications of this framework to layered networks encountered a vanishing gradient problem that has not yet been solved in a simple, biologically-plausible way. In this paper, we demonstrate that the vanishing gradient problem can be mitigated by replacing some of a layered network's connections with random layer-skipping connections in a manner inspired by small-world networks. This approach would be convenient to implement in neuromorphic hardware, and is biologically-plausible.
Citation
Frontiers in Neuroscience

Citation

Gammell, J. , Nam, S. and McCaughan, A. (2021), Layer-Skipping Connections Improve the Effectiveness of Equilibrium Propagation on Layered Networks, Frontiers in Neuroscience (Accessed June 19, 2024)

Issues

If you have any questions about this publication or are having problems accessing it, please contact reflib@nist.gov.

Created May 17, 2021, Updated June 16, 2023