학술논문

FAFusion: Learning for Infrared and Visible Image Fusion via Frequency Awareness
Document Type
Periodical
Source
IEEE Transactions on Instrumentation and Measurement IEEE Trans. Instrum. Meas. Instrumentation and Measurement, IEEE Transactions on. 73:1-11 2024
Subject
Power, Energy and Industry Applications
Components, Circuits, Devices and Systems
Image fusion
Feature extraction
Transformers
Frequency synthesizers
Data mining
Brightness
Deep learning
image fusion
frequency-aware
Language
ISSN
0018-9456
1557-9662
Abstract
In this article, we introduce a novel Frequency-Aware Infrared and Visible Image Fusion Network (FAFusion) designed to explore both low and high-frequency information present in infrared and visible images. Our approach involves leveraging the low-frequency information, which preserves the contour and overall brightness of the source images, in the encoder. This enables FAFusion to prioritize salient targets. Simultaneously, we feed the high-frequency information, preserving object edges and texture details, into the decoder, enhancing the fused images with richer details. To maintain the frequency information from the source images, we propose a maximum frequency loss function. This function considers the frequency components between the fused image and the source images, ensuring the preservation of critical frequency details. Additionally, we introduce a Residual Multiscale Feature Extraction (RMFE) module to capture diverse contextual information from the source images. The resulting FAFusion demonstrates the capability to generate fused images that exhibit both rich texture details and emphasize salient targets. We validate the effectiveness of our approach through extensive experiments on three publicly available datasets (TNO, MSRS, and M3FD). Comparative analyses against nine state-of-the-art methods highlight the superior visual effects, quantitative metrics, and generalization performance achieved by the proposed FAFusion.The source codes will be available at https://github.com/guobaoxiao/FAFusion.