26 December 2024 Multi-feature decomposition and transformer-fusion: an infrared and visible image fusion network based on multi-feature decomposition and transformer
Xujun Li, Zhicheng Duan, Jia Chang
Author Affiliations +
Abstract

Most of the existing infrared and visible image fusion methods focus only on the extraction of detail and basic features of both images. However, these methods ignore the extraction of shallow common features, which causes the problem of weakening the ability of detail and basic feature extraction. Consequently, we propose an infrared and visible image fusion network based on multi-feature decomposition and transformer (MFDT-Fusion). The multi-feature decomposition is processed by three encoders. First, to fully extract the shallow common features of the images, multi-head gated block based on transformer improvement is proposed as the common feature encoder (CFE). Multi-head dilated attention is designed in CFE to aggregate information from different receptive fields of infrared and visible images. Then, the long-range information-capturing ability of the transformer is used to compensate for the limited receptive field problem of the convolutional neural network. The detail feature encoder and basic feature encoder are designed to decompose the two features of the images. Finally, the features are separately fused by detail and basic fusion layers. Experimental results on three public datasets show that MFDT-Fusion achieves better performance than state-of-the-art methods.

© 2024 SPIE and IS&T
Xujun Li, Zhicheng Duan, and Jia Chang "Multi-feature decomposition and transformer-fusion: an infrared and visible image fusion network based on multi-feature decomposition and transformer," Journal of Electronic Imaging 33(6), 063053 (26 December 2024). https://doi.org/10.1117/1.JEI.33.6.063053
Received: 17 June 2024; Accepted: 10 December 2024; Published: 26 December 2024
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Image fusion

Infrared imaging

Infrared radiation

Visible radiation

Feature extraction

Transformers

Feature fusion

Back to Top