Multimodal sensor medical image fusion based on nonsubsampled shearlet transform and S-PCNNs in HSV space

Abstract

Abstract Computational imaging plays an important role in medical treatment for providing more comprehensive medical images. This work proposes a new scheme to fuse computed tomography (CT), magnetic resonance (MRI), and positron emission tomography (PET) images into a single image. A novel two-stage medical image fusion scheme, which is based on non-subsampled shearlet transform (NSST) and simplified pulse coupled neural networks (S-PCNNs), is proposed in the hue-saturation-value (HSV) color space. Firstly, CT and MRI images are decomposed into a set of low and high frequency coefficients by NSST, PET images are transformed into the HSV color space, and then the V component of PET image in the HSV color space. Secondly, intersecting cortical models (ICMs) are utilized to extract the edges and outlines in a larger area from the high frequency coefficients, and S-PCNNs are employed to describe the subtly detailed information in a smaller area. Thirdly, different fusion rules are designed to fuse the corresponding low and high frequency coefficients. At last, the fused medical image is obtained by the inverse HSV and NSST transformation, successively. The experimental results show that the proposed scheme is effective, and it can fuse more information into the final images than conventional methods.