Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations

Jiaqi He; Ruilai Wei; Shuaipeng Ge; Wenqiang Wu; Jianchao Guo; Juan Tao; Ru Wang; Chunfeng Wang; Caofeng Pan

doi:10.1002/inf2.12493

journal article Open Access Nov 05, 2023

Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations

Jiaqi He

Ruilai Wei Shuaipeng Ge Wenqiang Wu Jianchao Guo Juan Tao

Ru Wang

Chunfeng Wang

Caofeng Pan

InfoMat Vol. 6 No. 3 · Wiley

View at Publisher Save 10.1002/inf2.12493

Abstract

AbstractThe emulation of human multisensory functions to construct artificial perception systems is an intriguing challenge for developing humanoid robotics and cross‐modal human–machine interfaces. Inspired by human multisensory signal generation and neuroplasticity‐based signal processing, here, an artificial perceptual neuro array with visual‐tactile sensing, processing, learning, and memory is demonstrated. The neuromorphic bimodal perception array compactly combines an artificial photoelectric synapse network and an integrated mechanoluminescent layer, endowing individual and synergistic plastic modulation of optical and mechanical information, including short‐term memory, long‐term memory, paired pulse facilitation, and “learning‐experience” behavior. Sequential or superimposed visual and tactile stimuli inputs can efficiently simulate the associative learning process of “Pavlov's dog”. The fusion of visual and tactile modulation enables enhanced memory of the stimulation image during the learning process. A machine‐learning algorithm is coupled with an artificial neural network for pattern recognition, achieving a recognition accuracy of 70% for bimodal training, which is higher than that obtained by unimodal training. In addition, the artificial perceptual neuron has a low energy consumption of ∼20 pJ. With its mechanical compliance and simple architecture, the neuromorphic bimodal perception array has promising applications in large‐scale cross‐modal interactions and high‐throughput intelligent perceptions.image

Topics

No keywords indexed for this article. Browse by subject →

References

51

[1]

10.1126/sciadv.abe3996

[2]

10.1002/adma.201803637

[3]

10.1038/s41528-022-00202-7

[4]

10.1038/s41467-021-21319-9

[5]

10.1002/smll.202301186

[6]

10.1002/adma.202301924

[7]

10.1007/s40820-023-01035-3

[8]

10.1038/s41928-022-00868-x

[9]

10.1126/sciadv.abg9450

[10]

10.1038/s41467-021-22047-w

[11]

10.1002/sstr.202000029

[12]

10.1016/j.nanoen.2014.10.037

[13]

An artificial neural tactile sensing system

Sungwoo Chun, Jong-Seok Kim, Yongsang Yoo et al.

Nature Electronics 10.1038/s41928-021-00585-x

[14]

10.1002/aisy.201900090

[15]

10.1002/adma.202003014

[16]

10.1038/s41467-022-32827-7

[17]

10.1016/j.scib.2019.11.016

[18]

10.1016/j.scib.2021.12.014

[19]

10.1016/j.nanoen.2022.107171

[20]

10.1038/s41928-022-00859-y

[21]

10.1038/s41565-022-01095-3

[22]

10.1002/adma.202200380

[23]

10.1021/acsnano.1c08482

[24]

10.1021/acsnano.1c04298

[25]

10.1038/s41467-021-21890-1

[26]

Bioinspired mechano-photonic artificial synapse based on graphene/MoS 2 heterostructure

Jinyong Yu, Xixi Yang, Guoyun Gao et al.

Science Advances 10.1126/sciadv.abd9117

[27]

A bioinspired flexible organic artificial afferent nerve

Yeongin Kim, Alex Chortos, Wentao Xu et al.

Science 10.1126/science.aao0098

[28]

10.1002/adma.201801291

[29]

10.1038/s41467-020-18375-y

[30]

10.1038/s41467-020-15105-2

[31]

10.1038/s41467-021-21404-z

[32]

Neuromorphic sensorimotor loop embodied by monolithically integrated, low-voltage, soft e-skin

Weichen Wang, Yuanwen Jiang, Donglai Zhong et al.

Science 10.1126/science.ade0086

[33]

10.1016/j.nanoen.2022.107418

[34]

10.1021/acs.nanolett.1c01482

[35]

An interactive mouthguard based on mechanoluminescence-powered optical fibre sensors for bite-controlled device operation

Bo Hou, Luying Yi, Chao Li et al.

Nature Electronics 10.1038/s41928-022-00841-8

[36]

10.1021/acsnano.1c09779

[37]

10.1002/inf2.12250

[38]

10.1002/admt.202200757

[39]

10.1021/jacs.2c06724

[40]

Mechanoluminescence Rebrightening the Prospects of Stress Sensing: A Review

Yixi Zhuang, Rong-Jun Xie

Advanced Materials 10.1002/adma.202005925

[41]

10.1103/physrevb.84.115205

[42]

10.1002/adma.201700951

[43]

10.1002/adma.201906433

[44]

10.1002/adma.201907826

[45]

10.1038/nature03010

[46]

10.1002/adma.201805913

[47]

10.1038/ncomms6404

[48]

10.1021/acs.nanolett.0c04370

[49]

10.1002/adma.201907527

[50]

10.1002/adma.201907747

Showing 50 of 51 references

Cited By

82

Multisensory integration through high-efficiency neuromorphic hardware

Zezhuang Yi, Yuhui Xie · 2026

Nano Research

Biomimetic halide perovskite synapses for neuromorphic computing

Zhongwen Sun, Xuan Zhao · 2026

eScience

Electrochemiluminescent devices: Structural composition, preparation process and applications for flexible/stretchable displays

Huichen Xu, Caofeng Pan · 2025

Wearable Electronics

Neuromorphic devices assisted by machine learning algorithms

Ziwei Huo, Qijun Sun · 2025

International Journal of Extreme Ma...

Metrics

82

Citations

51

References

Details

Published: Nov 05, 2023
Vol/Issue: 6(3)
License: View

Authors

J

Jiaqi He

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China; School of Nanoscience and Engineering University of Chinese Academy of Sciences Beijing the People's Republic of China

R

Ruilai Wei

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China

S

Shuaipeng Ge

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China

W

Wenqiang Wu

Guangdong Research Center for Interfacial Engineering of Functional Materials, College of Materials Science and Engineering Shenzhen University Shenzhen the People's Republic of China

J

Jianchao Guo

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China; School of Nanoscience and Engineering University of Chinese Academy of Sciences Beijing the People's Republic of China

J

Juan Tao

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China

R

Ru Wang

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China; Guangdong Research Center for Interfacial Engineering of Functional Materials, College of Materials Science and Engineering Shenzhen University Shenzhen the People's Republic of China

C

Chunfeng Wang

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China; Guangdong Research Center for Interfacial Engineering of Functional Materials, College of Materials Science and Engineering Shenzhen University Shenzhen the People's Republic of China

C

Caofeng Pan

CAS Center for Excellence in Nanoscience, Beijing Key Laboratory of Micro‐nano Energy and Sensor Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences Beijing the People's Republic of China; School of Nanoscience and Engineering University of Chinese Academy of Sciences Beijing the People's Republic of China

Funding

National Natural Science Foundation of China Award: 52002246

Fundamental Research Funds for the Central Universities

Natural Science Foundation of Beijing Municipality Award: 2222088

National Key Research and Development Program of China Award: 2021YFB3200304

Cite This Article

Jiaqi He, Ruilai Wei, Shuaipeng Ge, et al. (2023). Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations. InfoMat, 6(3). https://doi.org/10.1002/inf2.12493

Artificial visual‐tactile perception array for enhanced memory and neuromorphic computations

You May Also Like