Gammell, J.
, Nam, S.
and McCaughan, A.
(2021),
Layer-Skipping Connections Improve the Effectiveness of Equilibrium Propagation on Layered Networks, Frontiers in Neuroscience
(Accessed September 23, 2023)
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
Secure .gov websites use HTTPS
A lock (
) or https:// means you’ve safely connected to the .gov website. Share sensitive information only on official, secure websites.
Layer-Skipping Connections Improve the Effectiveness of Equilibrium Propagation on Layered Networks
Published
Author(s)
, ,
Abstract
Equilibrium propagation is a learning framework that marks a step forward in the search for a biologically-plausible implementation of deep learning, and could be implemented efficiently in neuromorphic hardware. Previous applications of this framework to layered networks encountered a vanishing gradient problem that has not yet been solved in a simple, biologically-plausible way. In this paper, we demonstrate that the vanishing gradient problem can be mitigated by replacing some of a layered network's connections with random layer-skipping connections in a manner inspired by small-world networks. This approach would be convenient to implement in neuromorphic hardware, and is biologically-plausible.Citation
Frontiers in Neuroscience
Pub Type
Journals
Citation
Created May 17, 2021, Updated June 16, 2023