Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Vivienne Sze; Yu-Hsin Chen; Tien-Ju Yang; Joel S. Emer

doi:10.1109/jproc.2017.2761740

journal article Dec 01, 2017

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

Vivienne Sze Yu-Hsin Chen

Tien-Ju Yang Joel S. Emer

Proceedings of the IEEE Vol. 105 No. 12 pp. 2295-2329 · Institute of Electrical and Electronics Engineers (IEEE)

View at Publisher Save 10.1109/jproc.2017.2761740

Topics

No keywords indexed for this article. Browse by subject →

References

165

[1]

10.1109/icassp.2013.6639347

[2]

Long Short-Term Memory

Sepp Hochreiter, Jürgen Schmidhuber

Neural Computation 10.1162/neco.1997.9.8.1735

[3]

10.1109/icra.2017.7989182

[4]

levine "End-to-end training of deep visuomotor policies" J Mach Learn Res (2016)

[5]

mnih "Playing atari with deep reinforcement learning" NIPS Deep Learning Workshop (2013)

[6]

kaelbling "Reinforcement learning: A survey" J Artif Intell Res (1996) 10.1613/jair.301

[7]

hemsoth "The next wave of deep learning applications" (2016)

[8]

shalev-shwartz "Safe, multi-agent, reinforcement learning for autonomous driving" Proc NIPS Workshop Learn Inference Control Multi-Agent Syst (2016)

[9]

10.1109/icra.2016.7487175

[10]

10.1109/cvpr.2017.769

[11]

10.1117/1.jbo.21.9.094002

[12]

Convolutional neural network architectures for predicting DNA–protein binding

Haoyang Zeng, Matthew D. Edwards, Ge Liu et al.

Bioinformatics 10.1093/bioinformatics/btw255

[13]

wang Deep learning for identifying metastatic breast cancer (2016)

[14]

simonyan "Two-stream convolutional networks for action recognition in videos" Proc NIPS (2014)

[15]

collobert "Natural language processing (almost) from scratch" J Mach Learn Res (2011)

[16]

Deep Neural Networks for Acoustic Modeling in Speech Recognition: The Shared Views of Four Research Groups

Geoffrey Hinton, Li Deng, Dong Yu et al.

IEEE Signal Processing Magazine 10.1109/msp.2012.2205597

[17]

The human splicing code reveals new insights into the genetic determinants of disease

Hui Yuan Xiong, Babak Alipanahi, Leo J. Lee et al.

Science 10.1126/science.1254806

[18]

van den oord WaveNet A Generative Model for Raw Audio (2016)

[19]

standard "High bandwidth memory (HBM) DRAM" Proc JES (2013)

[20]

Predicting the sequence specificities of DNA- and RNA-binding proteins by deep learning

Babak Alipanahi, Andrew Delong, Matthew T Weirauch et al.

Nature Biotechnology 10.1038/nbt.3300

[21]

10.1109/vlsit.2012.6242474

[22]

Predicting effects of noncoding variants with deep learning–based sequence model

Jian Zhou, Olga G Troyanskaya

Nature Methods 10.1038/nmeth.3547

[23]

simonyan "Very deep convolutional networks for large-scale image recognition" Proc ICLR (2015)

[24]

Going deeper with convolutions

Christian Szegedy, Wei Liu, Yangqing Jia et al.

2015 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2015.7298594

[25]

lim Two-Dimensional Signal and Image Processing (1990)

[26]

mao "Exploring the regularity of sparse structure in convolutional neural networks" Proc CVPR Workshop Tensor Methods In Comput Vis (2017)

[27]

howard Mobilenets Efficient convolutional neural networks for mobile vision applications (2017)

[28]

Xception: Deep Learning with Depthwise Separable Convolutions

Francois Chollet

2017 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2017.195

[29]

wen "Learning structured sparsity in deep neural networks" Proc NIPS (2016)

[30]

10.1145/3140659.3080215

[31]

10.1145/3005348

[32]

DNN Energy Estimation

[33]

10.1145/2554688.2554785

[34]

10.1145/3007787.3001163

[35]

10.1145/3079856.3080254

[36]

TensorFlow-Slim Image Classification Library

[37]

Matconvnet Pretrained Models

[38]

Caffe Model Zoo

[39]

Caffe LeNet MNIST

[40]

urban "Do deep convolutional nets really need to be deep and convolutional?" Proc ICLR (2017)

[41]

szegedy "Inception-V4, inception-ResNet and the impact of residual connections on learning" Proc AAAI (2017)

[42]

Rethinking the Inception Architecture for Computer Vision

Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.308

[43]

lin "Network in network" Proc ICLR (2014)

[44]

nair "Rectified linear units improve restricted boltzmann machines" Proc ICML (2010)

[45]

The Benchmarking Process

[46]

romero "FitNets: Hints for thin deep nets" Proc ICLR (2015)

[47]

hinton "Distilling the knowledge in a neural network" NIPS Deep Learning Workshop (2014)

[48]

ba "Do deep nets really need to be deep?" Proc NIPS (2014)

[49]

bucilu? "Model compression" Proc SIGKDD (2006)

[50]

kim "Compression of deep convolutional neural networks for fast and low power mobile applications" Proc ICLR (2016)

Showing 50 of 165 references

Cited By

3,239

An All-Digital 8.6-nJ/Frame 65-nm Tsetlin Machine Image Classification Accelerator

Svein Anders Tunheim, Yujin Zheng · 2026

IEEE Transactions on Circuits and S...

A Wireless Charging Alignment Strategy Under Complex Working Conditions Using Voltage Fingerprints and a Residual-Calibrated Ensemble Network

Linxuan Li, Dechun Yuan · 2026

IEEE Access

A Review of Artificial Neural Network Applications in Proton Exchange Membrane Fuel Cells: Performance Prediction, Fault Diagnosis, and Degradation Modeling

Zhewei Yang, Guofeng Chang · 2026

Energy Technology

Interpretable rate-of-penetration prediction using a Bayesian-optimized MAML-LSTM-Transformer under small-sample conditions

Chao Min, Jialiang Ge · 2026

Energy and AI

A Transformer-Based Variational Autoencoder for Training Data Generation in Spindle Motor Vibration-Based Anomaly Detection

Jaeyoung Kim, Youngbae Hwang · 2026

Sensors

IMEA-Net: An edge-sensitive network for cropland change detection in high-resolution remote sensing images

Xiaoli Yuan, Li Chen · 2026

ISPRS Journal of Photogrammetry and...

Application of artificial intelligence methods in the international roughness index prediction of rigid and composite pavements: a systematic review

Hossein Hariri Asli, Nicholas Brake · 2026

Journal of Infrastructure Preservat...

Artificial intelligence and sustainable development: A global nonlinear analysis of the moderating roles of human capital and renewable energy

Chen Zhang, Rongrong Li · 2026

Renewable and Sustainable Energy Re...

A harmonized global prediction of biodegradable dissolved organic carbon in freshwater systems using optical indices and machine learning

Chae Eun Lee, Haeseong Oh · 2026

Environmental Research

Simulation of financial management data platform for big data enterprises based on computer network security detection algorithm

Lina Zhu, Zhe Qu · 2026

Results in Engineering

Energy-efficient machine learning approaches for enhanced biofuel (Biodiesel) production: A review

Myung-Kyu Song, Deok-Won Kim · 2026

Energy and AI

Physics-informed dual-stream networks for battery health estimation

Zhongyao Yang, Jinrui Nan · 2026

Results in Engineering

Breaking the DSP Wall: A Software–Hardware Co-Designed, Adaptive Error-Compensated MAC Architecture for Efficient Edge AI

Changyan Liu, Juntai Heiyan · 2026

Electronics

A Survey on Deep Learning Hardware Accelerators for Heterogeneous HPC Platforms

Cristina Silvano, Daniele Ielmini · 2025

ACM Computing Surveys

A Low-Latency Feed-Forward Architecture for Image Filtering via Row-by-Row Processing

Joe Gould, Ryan K. Nelson · 2025

IEEE Transactions on Circuits and S...

Edge Intelligence: A Review of Deep Neural Network Inference in Resource-Limited Environments

Dat Ngo, Hyun-Cheol Park · 2025

Electronics

Yield prediction in semiconductor manufacturing using two-step machine learning

Rebecca Busch, Peter Czerner · 2025

International Journal of Production...

Advances in UAV Path Planning: A Comprehensive Review of Methods, Challenges, and Future Directions

Wenlong Meng, Xuegang Zhang · 2025

Drones

Galley: Modern Query Optimization for Sparse Tensor Programs

Kyle Deeds, Willow Ahrens · 2025

Proceedings of the ACM on Managemen...

Quantitative analysis of watermelon fruit skin phenotypic traits via image processing and their potential in maturity and quality detection

Qing Gu, Tong Li · 2025

Computers and Electronics in Agricu...

Metrics

3,239

Citations

165

References

Details

Published: Dec 01, 2017
Vol/Issue: 105(12)
Pages: 2295-2329
License: View

Authors

Funding

MIT CICS

Cite This Article

Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, et al. (2017). Efficient Processing of Deep Neural Networks: A Tutorial and Survey. Proceedings of the IEEE, 105(12), 2295-2329. https://doi.org/10.1109/jproc.2017.2761740

Efficient Processing of Deep Neural Networks: A Tutorial and Survey

You May Also Like