MPCFusion: Multi-scale parallel cross fusion for infrared and visible images via convolution and vision Transformer

Haojie Tang; Yao Qian; Mengliang Xing; Yisheng Cao; Gang Liu

doi:10.1016/j.optlaseng.2024.108094

journal article May 01, 2024

MPCFusion: Multi-scale parallel cross fusion for infrared and visible images via convolution and vision Transformer

Haojie Tang

Yao Qian

Mengliang Xing Yisheng Cao

Gang Liu

Optics and Lasers in Engineering Vol. 176 pp. 108094 · Elsevier BV

View at Publisher Save 10.1016/j.optlaseng.2024.108094

Topics

No keywords indexed for this article. Browse by subject →

References

60

[1]

Bai "An object detection algorithm based on decision-level fusion of visible light image and infrared image" J Air Force Eng Univ Natural Sci Ed (2020)

[2]

Tang "Image fusion in the loop of high-level vision tasks: a semantic-aware real-time infrared and visible image fusion network" Inf Fusion (2022) 10.1016/j.inffus.2021.12.004

[3]

Awad "Adaptive near-infrared and visible fusion for fast image enhancement" IEEE Trans Comput Imaging (2020) 10.1109/tci.2019.2956873

[4]

Muller "Cognitively-engineered multisensor image fusion for military applications" Inf Fusion (2009) 10.1016/j.inffus.2008.08.008

[5]

Cao "Pedestrian detection with unsupervised multispectral feature learning using deep neural networks" Inf Fusion (2019) 10.1016/j.inffus.2018.06.005

[6]

Simone "Image fusion techniques for remote sensing applications" Inf Fusion (2002) 10.1016/s1566-2535(01)00056-2

[7]

Chen "Multi-focus image fusion based on multi-scale gradients and image matting" IEEE Trans Multimed (2022) 10.1109/tmm.2021.3057493

[8]

Wu "DMEF: multi-exposure image fusion based on a novel deep decomposition method" IEEE Trans Multimed (2022)

[9]

Xing "Multi-level adaptive perception guidance based infrared and visible image fusion" Opt Lasers Eng (2023) 10.1016/j.optlaseng.2023.107804

[10]

Du "Three-layer medical image fusion with tensor-based features" Inf Sci (2020) 10.1016/j.ins.2020.03.051

[11]

Ma "Infrared and visible image fusion methods and applications: a survey" Inf Fusion (2019) 10.1016/j.inffus.2018.02.004

[12]

Liu "A fusion algorithm for infrared and visible based on guided filtering and phase congruency in NSST domain" Opt Lasers Eng (2017) 10.1016/j.optlaseng.2017.05.007

[13]

Dong "FusionCPP: cooperative fusion of infrared and visible light images based on PCNN and PID control systems" Opt Lasers Eng (2024) 10.1016/j.optlaseng.2023.107821

[14]

Luo "Infrared and visible image fusion based on VPDE model and VGG network" Appl Intell (2023) 10.1007/s10489-023-04692-4

[15]

Tang "MdedFusion: a multi-level detail enhancement decomposition method for infrared and visible image fusion" Infrared Phys Technol (2022) 10.1016/j.infrared.2022.104435

[16]

Bavirisetti "Two-scale image fusion of visible and infrared images using saliency detection" Infrared Phys Technol (2016) 10.1016/j.infrared.2016.01.009

[17]

Zhang "Infrared and visible image fusion based on saliency detection and two-scale transform decomposition" Infrared Phys Technol (2021) 10.1016/j.infrared.2020.103626

[18]

Yang "Infrared and visible image fusion using visual saliency sparse representation and detail injection model" IEEE Trans Instrum Meas (2021) 10.1109/tim.2021.3126366

[19]

Qian "BTSFusion: fusion of infrared and visible image via a mechanism of balancing texture and salience" Opt Lasers Eng (2024) 10.1016/j.optlaseng.2023.107925

[20]

Zhang "SDNet: a versatile squeeze-and-decomposition network for real-time image fusion" Int J Comput Vis (2021) 10.1007/s11263-021-01501-8

[21]

Li "DANT-GAN: a dual attention-based of nested training network for infrared and visible image fusion" Digit Signal Process (2024) 10.1016/j.dsp.2023.104316

[22]

Ma "FusionGAN: a generative adversarial network for infrared and visible image fusion" Inf Fusion (2019) 10.1016/j.inffus.2018.09.004

[23]

Li "RFN-nest: an end-to-end residual fusion network for infrared and visible images" Inf Fusion (2021) 10.1016/j.inffus.2021.02.023

[24]

Li "Nestfuse: an infrared and visible image fusion architecture based on nest connection and spatial/channel attention models" IEEE Trans Instrum Meas (2020) 10.1109/tim.2020.3005230

[25]

Vs "Image fusion transformer" (2022)

[26]

Li "AttentionFGan: infrared and visible image fusion using attention-based generative adversarial networks" IEEE Trans Multimed (2020) 10.1109/tmm.2020.2997127

[27]

Dosovitskiy "An image is worth 16x16 words: transformers for image recognition at scale" (2020)

[28]

Tang "PIAFusion: A progressive infrared and visible image fusion network based on illumination aware" Inf Fusion (2022) 10.1016/j.inffus.2022.03.007

[29]

Xu "U2fusion: a unified unsupervised image fusion network" IEEE Trans Pattern Anal Mach Intell (2022) 10.1109/tpami.2020.3012548

[30]

Long "RXDNFuse: a aggregated residual dense network for infrared and visible image fusion" Inf Fusion (2021) 10.1016/j.inffus.2020.11.009

[31]

Xie "Aggregated residual transformations for deep neural networks" (2017)

[32]

Huang "Densely connected convolutional networks" (2017)

[33]

Zhao "MetaFusion: infrared and visible image fusion via meta-feature embedding from object detection" (2023)

[34]

Zhao "Correlation-driven dual-branch feature decomposition for multi-modality image fusion" (2023)

[35]

Liu "Coconet: Coupled contrastive learning network with multi-level feature ensemble for multi-modality image fusion" Int J Comput Vis (2023)

[36]

Tang "SuperFusion: a versatile image registration and fusion network with semantic awareness" IEEE/CAA J Autom Sin (2022) 10.1109/jas.2022.106082

[37]

Xu "MURF: mutually reinforcing multi-modal image registration and fusion" IEEE Trans Pattern Anal Mach Intell (2023) 10.1109/tpami.2023.3283682

[38]

Vaswani "Attention is all you need" Adv Neural Inf Process Syst (2017)

[39]

Wang "Res2Fusion: infrared and visible image fusion based on dense Res2net and double nonlocal attention models" IEEE Trans Instrum Meas (2022)

[40]

Wang "SwinFuse: a residual swin transformer fusion network for infrared and visible images" IEEE Trans Instrum Meas (2022) 10.1109/tim.2022.3216413

[41]

Ma "SwinFusion: cross-domain long-range learning for general image fusion via swin transformer" IEEE/CAA J Autom Sin (2022) 10.1109/jas.2022.105686

[42]

Goodfellow "Generative adversarial nets" Adv Neural Inf Process Syst (2014)

[43]

Wang "Infrared and visible image fusion via interactive compensatory attention adversarial learning" IEEE Trans Multimed (2022)

[44]

Guo "Convolutional neural networks meet vision transformers" (2022)

[45]

Sandler "Mobilenetv2: inverted residuals and linear bottlenecks" (2018)

[46]

Woo "CBAM: convolutional block attention module" (2018)

[47]

Image quality assessment: from error visibility to structural similarity

Zhou Wang, A.C. Bovik, H.R. Sheikh et al.

IEEE Transactions on Image Processing 2004 10.1109/tip.2003.819861

[48]

He "Guided image filtering" (2010)

[49]

Toet

[50]

Hwang "Multispectral pedestrian detection: benchmark dataset and baseline" (2015)

Showing 50 of 60 references

Cited By

27

PPIFuse: Physical Priors Injected Infrared and Visible Image Fusion

Qianhong Zhang, Qiao Liu · 2026

IEEE Transactions on Circuits and S...

WaveFusionNet: Infrared and visible image fusion based on multi-scale feature encoder–decoder and discrete wavelet decomposition

Renhe Liu, Yu Liu · 2024

Optics Communications

Metrics

27

Citations

60

References

Details

Published: May 01, 2024
Vol/Issue: 176
Pages: 108094
License: View

Authors

Metabolic Disease Research, Global Pharmaceutical Research and Discovery Organization, Abbott Laboratories, 100 Abbott Park Road, Abbott Park, Illinois 60064-6098

Cite This Article

Haojie Tang, Yao Qian, Mengliang Xing, et al. (2024). MPCFusion: Multi-scale parallel cross fusion for infrared and visible images via convolution and vision Transformer. Optics and Lasers in Engineering, 176, 108094. https://doi.org/10.1016/j.optlaseng.2024.108094

MPCFusion: Multi-scale parallel cross fusion for infrared and visible images via convolution and vision Transformer

You May Also Like