arXiv subjects

Guofen Wang

Publications and source records attributed to Guofen Wang.

3 recordsLinked to original sources

arXiv2025

DM-FNet: Unified multimodal medical image fusion via diffusion process-trained encoder-decoder

Multimodal medical image fusion (MMIF) extracts the most meaningful information from multiple source images, enabling a more comprehensive and accurate diagnosis. Achieving high-quality fusion results requires a careful balance of brightness, color, contrast, and detail; this ensures that the fused images effectively display relevant anatomical structures and reflect the functional status of the tissues. However, existing MMIF methods have limited capacity to capture detailed features during conventional training and suffer from insufficient cross-modal feature interaction, leading to suboptimal fused image quality. To address these issues, this study proposes a two-stage diffusion model-based fusion network (DM-FNet) to achieve unified MMIF. In Stage I, a diffusion process trains UNet for image reconstruction. UNet captures detailed information through progressive denoising and represents multilevel data, providing a rich set of feature representations for the subsequent fusion network. In Stage II, noisy images at various steps are input into the fusion network to enhance the model's feature recognition capability. Three key fusion modules are also integrated to process medical images from different modalities adaptively. Ultimately, the robust network structure and a hybrid loss function are integrated to harmonize the fused image's brightness, color, contrast, and detail, enhancing its quality and information density. The experimental results across various medical image types demonstrate that the proposed method performs exceptionally well regarding objective evaluation metrics. The fused image preserves appropriate brightness, a comprehensive distribution of radioactive tracers, rich textures, and clear edges. The code is available at https://github.com/HeDan-11/DM-FNet.

cs.CV↗

arXiv2024

Rethinking Normalization Strategies and Convolutional Kernels for Multimodal Image Fusion

Multimodal image fusion (MMIF) integrates information from different modalities to obtain a comprehensive image, aiding downstream tasks. However, existing research focuses on complementary information fusion and training strategies, overlooking the critical role of underlying architectural components like normalization and convolution kernels. We reevaluate the UNet architecture for end-to-end MMIF, identifying that widely used batch normalization limits performance by smoothing crucial sparse features. To address this, we propose a hybrid of instance and group normalization to maintain sample independence and reinforce intrinsic feature correlations. Crucially, this strategy facilitates richer feature maps, enabling large kernel convolution to fully leverage its receptive field, enhancing detail preservation. Furthermore, the proposed multi-path adaptive fusion module dynamically calibrates features from varying scales and receptive fields, ensuring effective information transfer. Our method achieves SOTA objective performance on MSRS, M$^3$FD, TNO, and Harvard datasets, producing visually clearer salient objects and lesion areas. Notably, it improves MSRS segmentation mIoU by 8.1\% over the infrared image. This performance stems from a synergistic design of normalization and convolution kernels, which preserves critical sparse features. The code is available at https://github.com/HeDan-11/LKC-FUNet.

cs.CV↗

arXiv2024

MCU-Net: A Multi-prior Collaborative Deep Unfolding Network with Gates-controlled Spatial Attention for Accelerated MR Image Reconstruction

Deep unfolding networks (DUNs) have demonstrated significant potential in accelerating magnetic resonance imaging (MRI). However, they often encounter high computational costs and slow convergence rates. Besides, they struggle to fully exploit the complementarity when incorporating multiple priors. In this study, we propose a multi-prior collaborative DUN, termed MCU-Net, to address these limitations. Our method features a parallel structure consisting of different optimization-inspired subnetworks based on low-rank and sparsity, respectively. We design a gates-controlled spatial attention module (GSAM), evaluating the relative confidence (RC) and overall confidence (OC) maps for intermediate reconstructions produced by different subnetworks. RC allocates greater weights to the image regions where each subnetwork excels, enabling precise element-wise collaboration. We design correction modules to enhance the effectiveness in regions where both subnetworks exhibit limited performance, as indicated by low OC values, thereby obviating the need for additional branches. The gate units within GSAMs are designed to preserve necessary information across multiple iterations, improving the accuracy of the learned confidence maps and enhancing robustness against accumulated errors. Experimental results on multiple datasets show significant improvements on PSNR and SSIM results with relatively low FLOPs compared to cutting-edge methods. Additionally, the proposed strategy can be conveniently applied to various DUN structures to enhance their performance.

eess.IV↗