Parallel convolutional processing using an integrated photonic tensor core

Batra, G., Jacobson, Z., Madhav, S., Queirolo, A. & Santhanam, N. Artificial-intelligence hardware: new opportunities for semiconductor companies. https://www.mckinsey.com/industries/semiconductors/our-insights/artificial-intelligence-hardware-new-opportunities-for-semiconductor-companies (McKinsey & Company, 2019).

[2]

Ben-Nun, T. & Hoefler, T. Demystifying parallel and distributed deep learning: an in-depth concurrency analysis. ACM Comput. Surv. 52, https://doi.org/10.1145/3320060 (2019). 10.1145/3320060

[3]

Herr, T. et al. Temporal solitons in optical microresonators. Nat. Photon. 8, 145–152 (2014). 10.1038/nphoton.2013.343

[4]

Herr, T., Gorodetsky, M. L. & Kippenberg, T. J. Dissipative Kerr solitons in optical microresonators. In Nonlinear Optical Cavity Dynamics From Microresonators to Fiber Lasers (ed. Grelu, P.) Vol. 8083, Ch. 6, 129–162 (Wiley, 2015). 10.1002/9783527686476.ch6

[5]

Raja, A. S. et al. Electrically pumped photonic integrated soliton microcomb. Nat. Commun. 10, 680 (2019).

[6]

Pfeiffer, M. H. P. et al. Photonic damascene process for integrated high-Q microresonator based nonlinear photonics. Optica 3, 20–25 (2016). 10.1364/optica.3.000020

[7]

Liu, J. et al. Ultralow-power chip-based soliton microcombs for photonic integration. Optica 5, 1347–1353 (2019). 10.1364/optica.5.001347

[8]

Machine Learning on AWS https://aws.amazon.com/machine-learning/ (accessed 12 October 2020).

[9]

Google Cloud AI And Machine Learning Products https://cloud.google.com/products/machine-learning/ (accessed 12 October 2020).

[10]

Zhang, C. et al. Optimizing FPGA-based Accelerator Design for Deep Convolutional Neural Networks. In ACM/SIGDA Int. Symp. Field-Programmable Gate Arrays (FPGA ’15) https://doi.org/10.1145/2684746.2689060 (2015). 10.1145/2684746.2689060

[11]

In-Datacenter Performance Analysis of a Tensor Processing Unit

Norman P. Jouppi, Cliff Young, Nishant Patil et al.

Proceedings of the 44th Annual International Sympo... 10.1145/3079856.3080246

[12]

Wang, P. S., Liu, Y., Guo, Y. X., Sun, C. Y. & Tong, X. O-CNN: octree-based convolutional neural networks for 3D shape analysis. ACM Trans. Graph. 36, https://doi.org/10.1145/3072959.3073608 (2017). 10.1145/3072959.3073608

[13]

Miller, D. A. B. Attojoule optoelectronics for low-energy information processing and communications. J. Lightwave Technol. 35, 346–396 (2017). 10.1109/jlt.2017.2647779

[14]

Agrawal, S. R. et al. A many-core architecture for in-memory data processing. In Proc. 50th Annu. IEEE/ACM Int. Symp. Microarchitecture (MICRO-50 ’17) 245–258, https://doi.org/10.1145/3123939.3123985 (IEEE/ACM, 2017). 10.1145/3123939.3123985

[15]

Are optical transistors the logical next step?

David A. B. Miller

Nature Photonics 2010 10.1038/nphoton.2009.240

[16]

In-memory computing with resistive switching devices

Daniele Ielmini, H.-S. Philip Wong

Nature Electronics 2018 10.1038/s41928-018-0092-2

[17]

Mixed-precision in-memory computing

Manuel Le Gallo, Abu Sebastian, Roland Mathis et al.

Nature Electronics 2018 10.1038/s41928-018-0054-8

[18]

Neuromorphic computing with multi-memristive synapses

Irem Boybat, Manuel Le Gallo, S. R. Nandakumar et al.

Nature Communications 2018 10.1038/s41467-018-04933-y

[19]

Sebastian, A., Le Gallo, M., Khaddam-Aljameh, R. & Eleftheriou, E. Memory devices and applications for in-memory computing. Nat. Nanotechnol. 15, 529–544 (2020). 10.1038/s41565-020-0655-z

[20]

Hu, M. et al. Dot-product engine for neuromorphic computing: programming 1T1M crossbar to accelerate matrix-vector multiplication. In Proc. 53rd Annu. Design Automation Conf. (DAC ’16) https://doi.org/10.1145/2897937.2898010 (ACM Digital Library, 2016). 10.1145/2897937.2898010

[21]

Gong, N. et al. Signal and noise extraction from analog memory elements for neuromorphic computing. Nat. Commun. 9, 2102 (2018).

[22]

Accurate deep neural network inference using computational phase-change memory

Vinay Joshi, Manuel Le Gallo, Simon Haefeli et al.

Nature Communications 2020 10.1038/s41467-020-16108-9

[23]

Yang, T. Y., Park, I. M., Kim, B. J. & Joo, Y. C. Atomic migration in molten and crystalline Ge2Sb2Te5 under high electric field. Appl. Phys. Lett. 95, 032104 (2009). 10.1063/1.3184584

[24]

Koelmans, W. W. et al. Projected phase-change memory devices. Nat. Commun. 6, 8181 (2015). 10.1038/ncomms9181

[25]

Kim, S. et al. A phase change memory cell with metallic surfactant layer as a resistance drift stabilizer. In 2013 IEEE Int. Electron Devices Meeting https://doi.org/10.1109/IEDM.2013.6724727 (IEEE, 2013). 10.1109/iedm.2013.6724727

[26]

Bell, T. E. Optical computing: a field in flux: a worldwide race is on to develop machines that compute with photons instead of electrons but what is the best approach? IEEE Spectr. 23, 34–38 (1986). 10.1109/mspec.1986.6371053

[27]

Hamerly, R., Bernstein, L., Sludds, A., Soljačić, M. & Englund, D. Large-scale optical neural networks based on photoelectric multiplication. Phys. Rev. X 9, 021032 (2018).

[28]

Silva, A. et al. Performing mathematical operations with metamaterials. Science 343, 160–163 (2014). 10.1126/science.1242818

[29]

Lin, X. et al. All-optical machine learning using diffractive deep neural networks. Science 361, 1004–1008 (2018). 10.1126/science.aat8084

[30]

Colburn, S., Chu, Y., Shilzerman, E. & Majumdar, A. Optical frontend for a convolutional neural network. Appl. Opt. 58, 3179–3186 (2019). 10.1364/ao.58.003179

[31]

Shen, Y. et al. Deep learning with coherent nanophotonic circuits. Nat. Photon. 11, 441–446 (2017). 10.1038/nphoton.2017.93

[32]

Tait, A. N. et al. Silicon photonic modulator neuron. Phys. Rev. Appl. 11, 064043 (2019). 10.1103/physrevapplied.11.064043

[33]

Pérez, D. et al. Multipurpose silicon photonics signal processor core. Nat. Commun. 8, 636 (2017).

[34]

Galal, S. & Horowitz, M. Energy-efficient floating-point unit design. IEEE Trans. Comput. 60, 913–922 (2011). 10.1109/tc.2010.121

[35]

Bangari, V. et al. Digital electronics and analog photonics for convolutional neural networks (DEAP-CNNs). IEEE J. Sel. Top. Quantum Electron. 26, https://doi.org/10.1109/JSTQE.2019.2945540 (2020). 10.1109/jstqe.2019.2945540

[36]

LeCun, Y., Cortes, C. & Borges, C. J. C. The MNIST database of handwritten digits. http://yann.lecun.com/exdb/mnist.

[37]

Stern, B., Ji, X., Okawachi, Y., Gaeta, A. L. & Lipson, M. Battery-operated integrated frequency comb generator. Nature 562, 401–405 (2018). 10.1038/s41586-018-0598-9

[38]

Jones, R. et al. Heterogeneously integrated InP/silicon photonics: fabricating fully functional transceivers. IEEE Nanotechnol. Mag. 13, 17–26 (2019). 10.1109/mnano.2019.2891369

[39]

Marin-Palomo, P. et al. Microresonator-based solitons for massively parallel coherent optical communications. Nature 546, 274–279 (2017). 10.1038/nature22387

[40]

Spencer, D. T. et al. An optical-frequency synthesizer using integrated photonics. Nature 557, 81–85 (2018). 10.1038/s41586-018-0065-7

[41]

Riemensberger, J. et al. Massively parallel coherent laser ranging using soliton microcombs. Nature 581, 164–170 (2019). 10.1038/s41586-020-2239-3

[42]

Moss, D. J., Morandotti, R., Gaeta, A. L. & Lipson, M. New CMOS-compatible platforms based on silicon nitride and Hydex for nonlinear optics. Nat. Photon. 7, 597–607 (2013). 10.1038/nphoton.2013.183

[43]

Deep Residual Learning for Image Recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren et al.

2016 IEEE Conference on Computer Vision and Patter... 10.1109/cvpr.2016.90

[44]

Simonyan, K. & Zisserman, A. Very deep convolutional networks for large-scale image recognition. In 3rd Int. Conf. Learning Representations (ICLR 2015) (eds Bengio, Y. & LeCun, Y.) 4 (2015); https://arxiv.org/abs/1409.1556.

[45]

Al-Ashrafy, M., Salem, A. & Anis, W. An efficient implementation of floating point multiplier. In 2011 Saudi Int. Electronics, Communications and Photonics Conf. (SIECPC) https://doi.org/10.1109/SIECPC.2011.5876905 (2011). 10.1109/siecpc.2011.5876905

[46]

Gao, L., Chen, P. Y. & Yu, S. Demonstration of convolution kernel operation on resistive cross-point array. IEEE Electron Device Lett. 37, 870–873 (2016). 10.1109/led.2016.2573140

[47]

Shafiee, A. et al. ISAAC: a convolutional neural network accelerator with in-situ analog arithmetic in crossbars. In Proc. 2016 43rd Int. Symp. Computer Architecture (ISCA 2016) https://doi.org/10.1109/ISCA.2016.12 (2016). 10.1109/isca.2016.12

[48]

Li, X. et al. Fast and reliable storage using a 5 bit, nonvolatile photonic memory cell. Optica 6, 1–6 (2019). 10.1364/optica.6.000001

[49]

Ríos, C. et al. Integrated all-photonic non-volatile multi-level memory. Nat. Photon. 9, 725–732 (2015). 10.1038/nphoton.2015.182

[50]

Feldmann, J. et al. Calculating with light using a chip-scale all-optical abacus. Nat. Commun. 8, 1256 (2017). 10.1038/s41467-017-01506-3

Showing 50 of 68 references

Cited By

1,465

Design and theoretical investigation of non-volatile reconfigurable and multi-wavelength integrated diffractive neural network on silicon

Changmao Deng, Wei Cheng · 2026

Optics & Laser Technology

Artificial intelligence with metasurfaces: from intelligent design to intelligent computing

Qian Ma, Zi Rui Feng · 2026

PhotoniX

Real-time aero-engine fault diagnosis using 5G edge computing and deep learning

Anping Wan, Zengzhen Zhu · 2026

Measurement

Nonlinear optical vector processing using linear silicon photonic circuits for 50 Gb/s memory and string similarity functions

T. Moschos, C. Pappas · 2025

Nature Communications

Single soliton microcomb combined with optical phased array for parallel FMCW LiDAR

Jingye Chen, Wenlei Li · 2025

Nature Communications

Parallel optical computing capable of 100-wavelength multiplexing

Ziqi Wei, Fangyuan Sha · 2025

eLight

Ultra-compact multi-task processor based on in-memory optical computing

Wencan Liu, Yuyao Huang · 2025

Light: Science & Applications

Ultrabroadband integrated electro-optic frequency comb in lithium tantalate

Junyin Zhang, Chengli Wang · 2025

Nature

A terahertz-bandwidth non-magnetic isolator

Haotian Cheng, Yishu Zhou · 2025

Nature Photonics

Hybrid Kerr-electro-optic frequency combs on thin-film lithium niobate

Yunxiang Song, Yaowen Hu · 2025

Light: Science & Applications

Coherently parallel fiber-optic distributed acoustic sensing using dual Kerr soliton microcombs

Jian-Ting Li, Bing Chang · 2024

Science Advances

Diffractive deep neural networks: Theories, optimization, and applications

Haijia Chen, Shaozhen Lou · 2024

Applied Physics Reviews

Integrated lithium niobate microwave photonic processing engine

Hanke Feng, Tong Ge · 2024

Nature

Perfect linear optics using silicon photonics

Miltiadis Moralis-Pegios, George Giamougiannis · 2024

Nature Communications

Optical neural networks: progress and challenges

Tingzhao Fu, Jianfa Zhang · 2024

Light: Science & Applications

Harnessing the capabilities of VCSELs: unlocking the potential for advanced integrated photonic devices and systems

Guanzhong Pan, Meng Xun · 2024

Light: Science & Applications

Inverse-designed low-index-contrast structures on a silicon photonics platform for vector–matrix multiplication

Vahid Nikkhah, Ali Pirmoradi · 2024

Nature Photonics

Diffractive optical computing in free space

Jingtian Hu, Deniz Mengu · 2024

Nature Communications

Contemporary implementations of spiking bio-inspired neural networks

Andrey Schegolev, Marina Bastrakova · 2024

Mesoscience & nanotechnology

Recent progress on inverse design for integrated photonic devices: methodology and applications

Ruoyu Shen, Bingzhou Hong · 2024

Journal of Nanophotonics

Metrics

1,465

Citations

68

References

Details

Published: Jan 06, 2021
Vol/Issue: 589(7840)
Pages: 52-58
License: View

Authors

University of Science and Technology of China

Jingdezhen Ceramic Institute

Cite This Article

J. Feldmann, N. Youngblood, M. Karpov, et al. (2021). Parallel convolutional processing using an integrated photonic tensor core. Nature, 589(7840), 52-58. https://doi.org/10.1038/s41586-020-03070-1

Parallel convolutional processing using an integrated photonic tensor core

You May Also Like