journal article
Open Access
Jul 16, 2024
A Comprehensive Review of Processing-in-Memory Architectures for Deep Neural Networks
Abstract
This comprehensive review explores the advancements in processing-in-memory (PIM) techniques and chiplet-based architectures for deep neural networks (DNNs). It addresses the challenges of monolithic chip architectures and highlights the benefits of chiplet-based designs in terms of scalability and flexibility. This review emphasizes dataflow-awareness, communication optimization, and thermal considerations in PIM-enabled manycore architectures. It discusses tailored dataflow requirements for different machine learning workloads and presents a heterogeneous PIM system for energy-efficient neural network training. Additionally, it explores thermally efficient dataflow-aware monolithic 3D (M3D) NoC architectures for accelerating CNN inferencing. Overall, this review provides valuable insights into the development and evaluation of chiplet and PIM architectures, emphasizing improved performance, energy efficiency, and inference accuracy in deep learning applications.
Topics
No keywords indexed for this article. Browse by subject →
References
50
[1]
Liu, J., Zhao, H., Ogleari, M.A., Li, D., and Zhao, J. (2018, January 20–24). Processing-in-Memory for Energy-Efficient Neural Network Training: A Heterogeneous Approach. Proceedings of the 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO-51), Fukuoka, Japan.
10.1109/micro.2018.00059
[2]
Sharma, H., Narang, G., Doppa, J.R., Ogras, U., and Pande, P.P. (2024). Dataflow-Aware PIM-Enabled Manycore Architecture for Deep Learning Workloads. arXiv, Available online: https://arxiv.org/abs/2403.19073.
[3]
Narang, G., Ogbogu, C., Doppa, J., and Pande, P. (2024). TEFLON: Thermally Efficient Dataflow-Aware 3D NoC for Accelerating CNN Inferencing on Manycore PIM Architectures. ACM Trans. Embed. Comput. Syst., just accepted.
10.1145/3665279
[4]
Joardar, B.K., Choi, W., Kim, R.G., Doppa, J.R., Pande, P.P., Marculescu, D., and Marculescu, R. (2017, January 19). 3D NoC-Enabled Heterogeneous Manycore Architectures for Accelerating CNN Training: Performance and Thermal Trade-Offs. Proceedings of the Eleventh IEEE/ACM International Symposium on Networks-on-Chip, Seoul, Republic of Korea.
10.1145/3130218.3130219
[5]
Giannoula, C., Yang, P., Vega, I.F., Yang, J., Li, Y.X., Luna, J.G., Sadrosadati, M., Mutlu, O., and Pekhimenko, G. (2024). Accelerating Graph Neural Networks on Real Processing-In-Memory Systems. arXiv.
[6]
Oliveira "Accelerating Neural Network Inference with Processing-in-DRAM: From the Edge to the Cloud" IEEE Micro (2022) 10.1109/mm.2022.3202350
[7]
Gómez-Luna, J., El Hajj, I., Fernandez, I., Giannoula, C., Oliveira, G.F., and Mutlu, O. (2021, January 18). Benchmarking Memory-Centric Computing Systems: Analysis of Real Processing-in-Memory Hardware. Proceedings of the 2021 12th International Green and Sustainable Computing Conference (IGSC), Pullman, WA, USA.
10.1109/igsc54211.2021.9651614
[8]
Ogbogu, C., Joardar, B.K., Chakrabarty, K., Doppa, J., and Pande, P.P. (2024). Data Pruning-enabled High Performance and Reliable Graph Neural Network Training on ReRAM-based Processing-in-Memory Accelerators. ACM Trans. Des. Autom. Electron. Syst., just accepted.
10.1145/3656171
[9]
Dhingra, P., Ogbogu, C., Joardar, B.K., Doppa, J.R., Kalyanaraman, A., and Pande, P.P. (2024). FARe: Fault-Aware GNN Training on Re-RAM-based PIM Accelerators. arXiv.
10.23919/date58400.2024.10546762
[10]
Lee, S., Kang, S.H., Lee, J., Kim, H., Lee, E., Seo, S., Yoon, H., Lee, S., Lim, K., and Shin, H. (2021, January 14–18). Hardware Architecture and Software Stack for PIM Based on Commercial DRAM Technology: Industrial Product. Proceedings of the 2021 ACM/IEEE 48th Annual International Symposium on Computer Architecture (ISCA), Valencia, Spain.
10.1109/isca52012.2021.00013
[11]
Joardar, B.K., Arka, A.I., Doppa, J.R., Pande, P.P., Li, H., and Chakrabarty, K. (2021, January 1). Heterogeneous Manycore Architectures Enabled by Processing-in-Memory for Deep Learning: From CNNs to GNNs (ICCAD Special Session Paper). Proceedings of the 2021 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), Munich, Germany.
10.1109/iccad51958.2021.9643559
[12]
Zheng, Q., Wang, Z., Feng, Z., Yan, B., Cai, Y., Huang, R., Chen, Y., Yang, C.L., and Li, H.H. (2020, January 20). Lattice: An ADC/DAC-less ReRAM-Based Processing-in-Memory Architecture for Accelerating Deep Convolutional Neural Networks. Proceedings of the 2020 57th ACM/IEEE Design Automation Conference (DAC), San Francisco, CA, USA.
10.1109/dac18072.2020.9218590
[13]
Zhao, X., Chen, S., and Kang, Y. (2024). Load Balanced PIM-Based Graph Processing. ACM Trans. Des. Autom. Electron. Syst., just accepted.
10.1145/3659951
[14]
Sharma "SWAP: A Server-Scale Communication-Aware Chiplet-Based Manycore PIM Accelerator" IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. (2022) 10.1109/tcad.2022.3197500
[15]
Jiang, H., Huang, S., Peng, X., and Yu, S. (2020, January 12). MINT: Mixed-Precision RRAM-Based In-Memory Training Architecture. Proceedings of the 2020 IEEE International Symposium on Circuits and Systems (ISCAS), Seville, Spain.
10.1109/iscas45731.2020.9181020
[16]
Das "Multi-Objective Hardware-Mapping Co-Optimisation for Multi-DNN Workloads on Chiplet-Based Accelerators" IEEE Trans. Comput. (2024) 10.1109/tc.2024.3386067
[17]
Hyun, B., Kim, T., Lee, D., and Rhu, M. (2024, January 2). Pathfinding Future PIM Architectures by Demystifying a Commercial PIM Technology. Proceedings of the 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), Edinburgh, UK.
10.1109/hpca57654.2024.00029
[18]
Lopes, A., Castro, D., and Romano, P. (2024, January 27). PIM-STM: Software Transactional Memory for Processing-In-Memory Systems. Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, La Jolla, CA, USA.
10.1145/3620665.3640428
[19]
Bavikadi, S., Sutradhar, P.R., Ganguly, A., and Dinakarrao, S.M.P. (2024, January 6–10). Reconfigurable Processing-in-Memory Architecture for Data Intensive Applications. Proceedings of the 2024 37th International Conference on VLSI Design and 2024 23rd International Conference on Embedded Systems (VLSID), Kolkata, India.
10.1109/vlsid60093.2024.00043
[20]
An, Y., Tang, Y., Yi, S., Peng, L., Pan, X., Sun, G., Luo, Z., Li, Q., and Zhang, J. (2024, January 2–6). StreamPIM: Streaming Matrix Computation in Racetrack Memory. Proceedings of the 2024 IEEE International Symposium on High-Performance Computer Architecture (HPCA), Edinburgh, UK.
10.1109/hpca57654.2024.00031
[21]
Gogineni, K., Dayapule, S.S., Gómez-Luna, J., Gogineni, K., Wei, P., Lan, T., Sadrosadati, M., Mutlu, O., and Venkataramani, G. (2024). SwiftRL: Towards Efficient Reinforcement Learning on Real Processing-In-Memory Systems. arXiv.
10.1109/ispass61541.2024.00029
[22]
Yang, Z., Ji, S., Chen, X., Zhuang, J., Zhang, W., Jani, D., and Zhou, P. (2024, January 22–25). Challenges and Opportunities to Enable Large-Scale Computing via Heterogeneous Chiplets. Proceedings of the 2024 29th Asia and South Pacific Design Automation Conference (ASP-DAC), Incheon, Republic of Korea.
10.1109/asp-dac58780.2024.10473961
[23]
Wang "Social Media Platform-Oriented Topic Mining and Information Security Analysis by Big Data and Deep Convolutional Neural Network" Technol. Forecast. Soc. Chang. (2024) 10.1016/j.techfore.2023.123070
[24]
Rego "Deep Learning Applications on Cybersecurity: A Practical Approach" Neurocomputing (2024) 10.1016/j.neucom.2023.126904
[25]
Akgun "A Novel Cyber Security Model Using Deep Transfer Learning" Arab. J. Sci. Eng. (2024) 10.1007/s13369-023-08092-1
[26]
Kasarapu, S., Bavikadi, S., and Dinakarrao, S.M. (2024). Empowering Malware Detection Efficiency within Processing-in-Memory Architecture. arXiv.
[27]
Kanellopoulos, K., Bostanci, F., Olgun, A., Yaglikci, A.G., Yuksel, I.E., Ghiasi, N.M., Bingol, Z., Sadrosadati, M., and Mutlu, O. (2024). Amplifying Main Memory-Based Timing Covert and Side Channels using Processing-in-Memory Operations. arXiv.
[28]
Asad, A., Kaur, R., and Mohammadi, F. (2022). A Survey on Memory Subsystems for Deep Neural Network Accelerators. Future Internet, 14.
10.3390/fi14050146
[29]
Kaur, R., and Mohammadi, F. (2023, January 5–8). Power Estimation and Comparison of Heterogeneous CPU-GPU Processors. Proceedings of the 2023 IEEE 25th Electronics Packaging Technology Conference (EPTC), Singapore.
10.1109/eptc59621.2023.10457590
[30]
Kaur, R., and Mohammadi, F. (2023, January 24–27). Comparative Analysis of Power Efficiency in Heterogeneous CPU-GPU Processors. Proceedings of the 2023 Congress in Computer Science, Computer Engineering, & Applied Computing (CSCE), Las Vegas, NV, USA.
10.1109/csce60160.2023.00129
[31]
Kaur, R., and Saluja, N. (2018, January 7–9). Comparative Analysis of 1-bit Memory Cell in CMOS and QCA Technology. Proceedings of the 2018 International Flexible Electronics Technology Conference (IFETC), Ottawa, ON, Canada.
10.1109/ifetc.2018.8584033
[32]
Safayenikoo, P., Asad, A., Fathy, M., and Mohammadi, F. (2017, January 14–15). An Energy Efficient Non-Uniform Last Level Cache Architecture in 3D Chip-Multiprocessors. Proceedings of the 2017 18th International Symposium on Quality Electronic Design (ISQED), Santa Clara, CA, USA.
10.1109/isqed.2017.7918344
[33]
Asad, A., AL-Obaidy, F., and Mohammadi, F. (2020, January 25–28). Efficient Power Consumption using Hybrid Emerging Memory Technology for 3D CMPs. Proceedings of the 2020 IEEE 11th Latin American Symposium on Circuits & Systems (LASCAS), San Jose, Costa Rica.
10.1109/lascas45839.2020.9069014
[34]
Asad, A., Kaur, R., and Mohammadi, F. (2022, January 14–16). Noise Suppression Using Gated Recurrent Units and Nearest Neighbor Filtering. Proceedings of the 2022 International Conference on Computational Science and Computational Intelligence (CSCI), Las Vegas, NV, USA.
10.1109/csci58124.2022.00072
[35]
Shin, H., Kang, M., and Kim, L. (2020, January 2–5). A thermal-aware optimization framework for ReRAM-based deep neural network acceleration. Proceedings of the ICCAD ’20: IEEE/ACM International Conference on Computer-Aided Design, Virtual Event, USA.
10.1145/3400302.3415665
[36]
Mutlu, O., Ghose, S., Gómez-Luna, J., and Ausavarungnirun, R. (2022). A modern primer on processing in memory. Emerging Computing: From Devices to Systems: Looking beyond Moore and Von Neumann, Springer Nature Singapore.
10.1007/978-981-16-7487-7_7
[37]
Yu "Multipim: A detailed and configurable multi-stack processing-in-memory simulator" IEEE Comput. Archit. Lett. (2021) 10.1109/lca.2021.3061905
[38]
Mosanu, S., Sakib, M.N., Tracy, T., Cukurtas, E., Ahmed, A., Ivanov, P., Khan, S., Skadron, K., and Stan, M. (2022, January 14–23). Pimulator: A fast and flexible processing-in-memory emulation platform. Proceedings of the 2022 Design Automation & Test in Europe Conference & Exhibition (DATE), Antwerp, Belgium.
10.23919/date54114.2022.9774614
[39]
Roesch, J., Lyubomirsky, S., Weber, L., Pollock, J., Kirisame, M., Chen, T., and Tatlock, Z. (2018, January 18). Relay: A new IR for machine learning frameworks. Proceedings of the 2nd ACM SIGPLAN International Workshop on Machine Learning and Programming Languages, Philadelphia, PA, USA.
10.1145/3211346.3211348
[40]
Kim "Silent-PIM: Realizing the processing-in-memory computing with standard memory requests" IEEE Trans. Parallel Distrib. Syst. (2021) 10.1109/tpds.2021.3065365
[41]
Jin "ReHy: A ReRAM-Based Digital/Analog Hybrid PIM Architecture for Accelerating CNN Training" IEEE Trans. Parallel Distrib. Syst. (2022)
[42]
Deng "Deep learning: Methods and applications" Found. Trends Signal Process (2014) 10.1561/2000000039
[43]
Wald "Not in name alone: A memristive memory processing unit for real in-memory processing" IEEE Micro (2018) 10.1109/mm.2018.053631137
[44]
Ronen "SIMPLER MAGIC: Synthesis and mapping of in-memory logic executed in a single row to improve throughput" IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. (2020) 10.1109/tcad.2019.2931188
[45]
Mittal "A survey of ReRAM-based architectures for processing-in-memory and neural networks" Mach. Learn. Knowl. Extr. (2019) 10.3390/make1010005
[46]
Kim "DeepTrain: A Programmable Embedded Platform for Training Deep Neural Networks" IEEE Trans. Comput.-Aided Des. Integr. Circuits Syst. (2018) 10.1109/tcad.2018.2858358
[47]
Boroumand, A., Ghose, S., Oliveira, G.F., and Mutlu, O. (2021). Polynesia: Enabling Effective Hybrid Transactional/Analytical Databases with Specialized Hardware/Software Co-Design. arXiv.
10.1109/icde53745.2022.00270
[48]
Gu, P., Xie, X., Ding, Y., Chen, G., Zhang, W., Niu, D., and Xie, Y. (June, January 30). iPIM: Programmable In-Memory Image Processing Accelerator using Near-Bank Architecture. Proceedings of the 2020 ACM/IEEE 47th Annual International Symposium on Computer Architecture (ISCA), Valencia, Spain.
10.1109/isca45697.2020.00071
[49]
Huang, Y., Zheng, L., Yao, P., Zhao, J., Liao, X., Jin, H., and Xue, J. (2020, January 18–22). A Heterogeneous PIM Hardware-Software Co-Design for Energy-Efficient Graph Processing. Proceedings of the 2020 IEEE International Parallel and Distributed Processing Symposium (IPDPS), New Orleans, LA, USA.
10.1109/ipdps47924.2020.00076
[50]
Liu "A survey of deep neural network architectures and their applications" Neurocomputing (2017) 10.1016/j.neucom.2016.12.038
Metrics
16
Citations
50
References
Details
- Published
- Jul 16, 2024
- Vol/Issue
- 13(7)
- Pages
- 174
- License
- View
Authors
Funding
NSERC
Cite This Article
Rupinder Kaur, Arghavan Asad, Fazel Mohammadi (2024). A Comprehensive Review of Processing-in-Memory Architectures for Deep Neural Networks. Computers, 13(7), 174. https://doi.org/10.3390/computers13070174
Related
You May Also Like
Understanding of Machine Learning with Deep Learning: Architectures, Workflow, Applications and Future Directions
Mohammad Mustafa Taye · 2023
801 citations
Impact of the Implementation of ChatGPT in Education: A Systematic Review
Marta Montenegro-Rueda, José Fernández-Cerero · 2023
362 citations
The YOLO Framework: A Comprehensive Review of Evolution, Applications, and Benchmarks in Object Detection
Momina Liaqat Ali, Zhou Zhang · 2024
253 citations
IoT-Enabled Soil Nutrient Analysis and Crop Recommendation Model for Precision Agriculture
Murali Krishna Senapaty, Abhishek Ray · 2023
155 citations
A Systematic Survey on Cybersickness in Virtual Environments
Ananth N. Ramaseri Chandra, Fatima El Jamiy · 2022
123 citations