A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Yuchen Wang; Malu Zhang; Xiaoling Luo; Hong Qu

doi:10.1016/j.neunet.2024.106244

journal article Jun 01, 2024

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

Yuchen Wang

Malu Zhang

Xiaoling Luo

Hong Qu

Neural Networks Vol. 174 pp. 106244 · Elsevier BV

View at Publisher Save 10.1016/j.neunet.2024.106244

Topics

No keywords indexed for this article. Browse by subject →

References

58

[1]

TrueNorth: Design and Tool Flow of a 65 mW 1 Million Neuron Programmable Neurosynaptic Chip

Filipp Akopyan, Jun Sawada, Andrew Cassidy et al.

IEEE Transactions on Computer-Aided Design of Inte... 2015 10.1109/tcad.2015.2474396

[2]

Brown "Language models are few-shot learners" (2020)

[3]

Bu "Optimized potential initialization for low-latency spiking neural networks" (2022)

[4]

Bu (2023)

[5]

Cao "Spiking deep convolutional neural networks for energy-efficient object recognition" International Journal of Computer Vision (2015) 10.1007/s11263-014-0788-3

[6]

Cassidy "Cognitive computing building block: A versatile and efficient digital neuron model for neurosynaptic cores" (2013)

[7]

AutoAugment: Learning Augmentation Strategies From Data

Ekin D. Cubuk, Barret Zoph, Dandelion Mané et al.

2019 IEEE/CVF Conference on Computer Vision and Pa... 10.1109/cvpr.2019.00020

[8]

Loihi: A Neuromorphic Manycore Processor with On-Chip Learning

Mike Davies, Narayan Srinivasa, Tsung-Han Lin et al.

IEEE Micro 2018 10.1109/mm.2018.112130359

[9]

Deng "Imagenet: A large-scale hierarchical image database" (2009)

[10]

Deng (2021)

[11]

Deng "Rethinking the performance comparison between SNNS and ANNS" Neural Networks (2020) 10.1016/j.neunet.2019.09.005

[12]

Devlin (2018)

[13]

DeVries (2017)

[14]

Diehl "Fast-classifying, high-accuracy spiking deep networks through weight and threshold balancing" (2015)

[15]

Ding, J., Yu, Z., Tian, Y., & Huang, T. (2021). Optimal ANN-SNN Conversion for Fast and Accurate Inference in Deep Spiking Neural Networks. In Proceedings of the thirtieth international joint conference on artificial intelligence (pp. 2328–2336). 10.24963/ijcai.2021/321

[16]

Fu "A novel deep learning-based collaborative filtering model for recommendation system" IEEE Transactions on Cybernetics (2018) 10.1109/tcyb.2018.2795041

[17]

Furber "The Spinnaker project" Proceedings of the IEEE (2014) 10.1109/jproc.2014.2304638

[18]

Han "Deep spiking neural network: Energy efficiency through time based coding" (2020)

[19]

Han, B., Srinivasan, G., & Roy, K. (2020). Rmp-snn: Residual membrane potential neuron for enabling deeper high-accuracy and low-latency spiking neural network. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 13558–13567). 10.1109/cvpr42600.2020.01357

[20]

Han "Symmetric-threshold ReLU for fast and nearly lossless ANN-SNN conversion" Machine Intelligence Research (2023) 10.1007/s11633-022-1388-2

[21]

Hao (2023)

[22]

He, K., Chen, X., Xie, S., Li, Y., Dollár, P., & Girshick, R. (2022). Masked autoencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 16000–16009). 10.1109/cvpr52688.2022.01553

[23]

He, X., Liao, L., Zhang, H., Nie, L., Hu, X., & Chua, T.-S. (2017). Neural collaborative filtering. In Proceedings of the 26th international conference on world wide web (pp. 173–182). 10.1145/3038912.3052569

[24]

Delving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2015 IEEE International Conference on Computer Vis... 10.1109/iccv.2015.123

[25]

Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.90

[26]

Ho "TCL: An ANN-to-SNN conversion with trainable clipping layers" (2021)

[27]

Horowitz "1.1 Computing’s energy problem (and what we can do about it)" (2014)

[28]

Simple model of spiking neurons

E.M. Izhikevich

IEEE Transactions on Neural Networks 2003 10.1109/tnn.2003.820440

[29]

Krizhevsky (2009)

[30]

Communication in Neuronal Networks

Simon B. Laughlin, Terrence J. Sejnowski

Science 2003 10.1126/science.1089662

[31]

Li "A free lunch from ANN: Towards efficient, accurate spiking neural networks calibration" (2021)

[32]

Li (2022)

[33]

Li "Differentiable spike: Rethinking gradient-descent for training spiking neural networks" Advances in Neural Information Processing Systems (2021)

[34]

Networks of spiking neurons: The third generation of neural network models

Wolfgang Maass

Neural Networks 1997 10.1016/s0893-6080(97)00011-7

[35]

Meng "Training much deeper spiking neural networks with a small number of time-steps" Neural Networks (2022) 10.1016/j.neunet.2022.06.001

[36]

Surrogate Gradient Learning in Spiking Neural Networks: Bringing the Power of Gradient-Based Optimization to Spiking Neural Networks

Emre O. Neftci, Hesham Mostafa, Friedemann Zenke

IEEE Signal Processing Magazine 2019 10.1109/msp.2019.2931595

[37]

Painkras "SpiNNaker: A 1-W 18-core system-on-chip for massively-parallel neural network simulation" IEEE Journal of Solid-State Circuits (2013) 10.1109/jssc.2013.2259038

[38]

Towards artificial general intelligence with hybrid Tianjic chip architecture

Jing Pei, Lei Deng, Sen Song et al.

Nature 2019 10.1038/s41586-019-1424-8

[39]

Rathi "DIET-SNN: A low-latency spiking neural network with direct input encoding and leakage and threshold optimization" IEEE Transactions on Neural Networks and Learning Systems (2021)

[40]

Rathi, N., Srinivasan, G., Panda, P., & Roy, K. (2020). Enabling Deep Spiking Neural Networks with Hybrid Conversion and Spike Timing Dependent Backpropagation. In 8th international conference on learning representations.

[41]

Towards spike-based machine intelligence with neuromorphic computing

Kaushik Roy, Akhilesh Jaiswal, Priyadarshini Panda

Nature 2019 10.1038/s41586-019-1677-2

[42]

Rueckauer "Conversion of continuous-valued deep networks to efficient event-driven networks for image classification" Frontiers in Neuroscience (2017) 10.3389/fnins.2017.00682

[43]

Shen "ESL-SNNs: An evolutionary structure learning strategy for spiking neural networks" (2023)

[44]

Silver "A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play" Science (2018) 10.1126/science.aar6404

[45]

Mastering the game of Go without human knowledge

David Silver, Julian Schrittwieser, Karen Simonyan et al.

Nature 2017 10.1038/nature24270

[46]

Simonyan (2014)

[47]

Stanojevic "An exact mapping from ReLU networks to spiking neural networks" Neural Networks (2023) 10.1016/j.neunet.2023.09.011

[48]

Suetake "S3NN: Time step reduction of spiking surrogate gradients for training energy efficient single-step spiking neural networks" Neural Networks (2023) 10.1016/j.neunet.2022.12.008

[49]

Deep learning in spiking neural networks

Amirhossein Tavanaei, Masoud Ghodrati, Saeed Reza Kheradpisheh et al.

Neural Networks 2019 10.1016/j.neunet.2018.12.002

[50]

Wang "Supervised learning in spiking neural networks: A review of algorithms and evaluations" Neural Networks (2020) 10.1016/j.neunet.2020.02.011

Showing 50 of 58 references

Metrics

27

Citations

58

References

Details

Published: Jun 01, 2024
Vol/Issue: 174
Pages: 106244
License: View

Authors

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China

X

Xiaoling Luo

Shenzhen Key Laboratory of Visual Object Detection and Recognition Harbin Institute of Technology Shenzhen China

H

Hong Qu

School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China

Funding

National Natural Science Foundation of China Award: 62236007

Sichuan Province Science and Technology Support Program Award: 2023YFG0259

Cite This Article

Yuchen Wang, Malu Zhang, Xiaoling Luo, et al. (2024). A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks. Neural Networks, 174, 106244. https://doi.org/10.1016/j.neunet.2024.106244

A universal ANN-to-SNN framework for achieving high accuracy and low latency deep Spiking Neural Networks

You May Also Like