Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Emre O. Neftci; Hesham Mostafa; Friedemann Zenke

doi:10.1109/msp.2019.2931595

journal article Nov 01, 2019

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Emre O. Neftci

Hesham Mostafa

Friedemann Zenke

IEEE Signal Processing Magazine Vol. 36 No. 6 pp. 51-63 · Institute of Electrical and Electronics Engineers (IEEE)

View at Publisher Save 10.1109/msp.2019.2931595

Topics

No keywords indexed for this article. Browse by subject →

References

41

[1]

10.1016/j.neunet.2016.07.006

[2]

nøkland "Direct feedback alignment provides learning in deep neural networks" Advances in Neural IInformation Processing Systems (2016)

[3]

Error-backpropagation in temporally encoded networks of spiking neurons

Sander M. Bohté, Joost N. Kok, Han La Poutré

Neurocomputing 10.1016/s0925-2312(01)00658-0

[4]

10.3389/fnins.2016.00508

[5]

10.3389/fnins.2017.00324

[6]

hunsberger "Spiking deep networks with LIF neurons" (2015)

[7]

Random synaptic feedback weights support error backpropagation for deep learning

Timothy P. Lillicrap, Daniel Cownden, Douglas B. Tweed et al.

Nature Communications 10.1038/ncomms13276

[8]

woz´niak "Deep networks incorporating spiking neural dynamics" (2018)

[9]

o’connor "Temporally efficient deep learning with spikes" (2017)

[10]

mostafa "Supervised learning based on temporal coding in spiking neural networks" IEEE Trans Neural Netw Learn Syst (2018)

[11]

10.1038/s41467-017-01827-3

[12]

10.3389/fnins.2018.00608

[13]

10.1109/mcse.2017.33

[14]

10.1017/cbo9781107447615

[15]

shrestha "SLAYER: Spike layer error reassignment in time" Advances in Neural IInformation Processing Systems (2018)

[16]

A Learning Algorithm for Continually Running Fully Recurrent Neural Networks

Ronald J. Williams, David Zipser

Neural Computation 10.1162/neco.1989.1.2.270

[17]

bellec "Biologically inspired alternatives to backpropagation through time for learning in recurrent neural nets" (2019)

[18]

courbariaux "Binarized neural networks: Training deep neural networks with weights and activations constrained to +1 or ?1" (2016)

[19]

bengio "Estimating or propagating gradients through stochastic neurons for conditional computation" (2013)

[20]

10.1073/pnas.1604850113

[21]

10.1007/978-3-642-21735-7_8

[22]

10.3389/fncom.2014.00038

[23]

kaiser "Synaptic plasticity for deep continuous local learning" (2018)

[24]

10.1523/jneurosci.4098-12.2013

[25]

bellec "Long short-term memory and learning-to-learn in networks of spiking neurons" Advances in Neural IInformation Processing Systems (2018)

[26]

10.1016/j.conb.2014.01.004

[27]

10.1162/neco_a_01080

[28]

Deep learning in spiking neural networks

Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh et al.

Neural Networks 10.1016/j.neunet.2018.12.002

[29]

10.1109/ijcnn.2015.7280618

[30]

10.1016/j.neuron.2014.03.026

[31]

10.1162/neco_a_01086

[32]

10.7554/elife.28295

[33]

goodfellow Deep Learning (2016)

[34]

shrestha "SLAYER: Spike layer error reassignment in time" Advances in neural information processing systems (2018)

[35]

huh "Gradient descent for spiking neural networks" Advances in Neural IInformation Processing Systems (2018)

[36]

10.1038/nn.4241

[37]

10.1162/neco.2006.18.6.1318

[38]

10.1038/s41467-017-01827-3

[39]

A Learning Algorithm for Boltzmann Machines*

David H. Ackley, Geoffrey E. Hinton, Terrence J. Sejnowski

Cognitive Science 10.1207/s15516709cog0901_7

[40]

10.7554/elife.22901

[41]

10.1162/neco_a_00790

Cited By

1,262

Randomized forward mode gradient for spiking neural networks in scientific machine learning

Ruyin Wan, Qian Zhang · 2026

Neural Networks

SNN-SC: A Spiking Semantic Communication Framework for Collaborative Intelligence

Mengyang Wang, Mengyao Ma · 2025

IEEE Transactions on Vehicular Tech...

Spiking neural networks for physiological and speech signals: a review

Sung Soo Park, Young-Seok Choi · 2024

Biomedical Engineering Letters

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Yuchen Wang, Malu Zhang · 2024

Neural Networks

A quantum leaky integrate-and-fire spiking neuron and network

Dean Brand, Francesco Petruccione · 2024

npj Quantum Information

Approximating Nonlinear Functions With Latent Boundaries in Low-Rank Excitatory-Inhibitory Spiking Networks

William F. Podlaski, Christian K. Machens · 2024

Neural Computation

A solution to the learning dilemma for recurrent networks of spiking neurons

Guillaume Bellec, Franz Scherr · 2020

Nature Communications

Metrics

1,262

Citations

41

References

Details

Published: Nov 01, 2019
Vol/Issue: 36(6)
Pages: 51-63
License: View

Authors

E

Emre O. Neftci

Department of Computer Science and the Department of Cognitive Sciences, University of California at Irvine (UC Irvine), Irvine, CA, USA

Funding

Wellcome Trust Award: 110124/Z/15/Z]

National Science Foundation Award: 1640081

Intel Corporation

Cite This Article

Emre O. Neftci, Hesham Mostafa, Friedemann Zenke (2019). Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks. IEEE Signal Processing Magazine, 36(6), 51-63. https://doi.org/10.1109/msp.2019.2931595

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

You May Also Like