Image Decomposition Approach Research Articles

Advancing multimodal medical image fusion: an adaptive image decomposition approach based on multilevel Guided filtering.

With the rapid development of medical imaging methods, multimodal medical image fusion techniques have caught the interest of researchers. The aim is to preserve information from diverse sensors using various models to generate a single informative image. The main challenge is to derive a trade-off between the spatial and spectral qualities of the resulting fused image and the computing efficiency. This article proposes a fast and reliable method for medical image fusion depending on multilevel Guided edge-preserving filtering (MLGEPF) decomposition rule. First, each multimodal medical image was divided into three sublayer categories using an MLGEPF decomposition scheme: small-scale component, large-scale component and background component. Secondly, two fusion strategies-pulse-coupled neural network based on the structure tensor and maximum based-are applied to combine the three types of layers, based on the layers' various properties. The three different types of fused sublayers are combined to create the fused image at the end. A total of 40 pairs of brain images from four separate categories of medical conditions were tested in experiments. The pair of images includes various case studies including magnetic resonance imaging (MRI) , TITc, single-photon emission computed tomography (SPECT) and positron emission tomography (PET). We included qualitative analysis to demonstrate that the visual contrast between the structure and the surrounding tissue is increased in our proposed method. To further enhance the visual comparison, we asked a group of observers to compare our method's outputs with other methods and score them. Overall, our proposed fusion scheme increased the visual contrast and received positive subjective review. Moreover, objective assessment indicators for each category of medical conditions are also included. Our method achieves a high evaluation outcome on feature mutual information (FMI), the sum of correlation of differences (SCD), Qabf and Qy indexes. This implies that our fusion algorithm has better performance in information preservation and efficient structural and visual transferring.

Self-supervised multi-task learning framework for safety and health-oriented road environment surveillance based on connected vehicle visual perception

Cutting-edge connected vehicle (CV) technologies have drawn much attention in recent years. The real-time traffic data captured by a CV can be shared with other CVs and data centers so as to open new possibilities for solving diverse transportation problems. The trajectory data of CVs have been well-studied and widely used. However, image data captured by onboard cameras in a connected environment, as being a kind of fundamental data source, are not sufficiently investigated, especially for safety and health-oriented visual perception. In this paper, a bidirectional process of image synthesis and decomposition (BPISD) approach is proposed, and thus a novel self-supervised multi-task learning framework, to simultaneously estimate depth map, atmospheric visibility, airlight, and PM2.5 mass concentration, in which depth map and visibility are considered highly associated with traffic safety, while airlight and PM2.5 mass concentration are directly correlated with human health. Both the training and testing phases of the proposed system solely require a single image as input. Due to the innovative training pipeline, the depth estimation network can automatically manage various levels of visibility conditions and overcome diverse inherent problems in current image-synthesis-based self-supervised depth estimation, thereby generating high-quality depth maps even in low-visibility situations and further benefiting accurate estimations of visibility, airlight, and PM2.5 mass concentration. Extensive experiments on the original and synthesized data from the KITTI dataset and real-world data collected in Beijing demonstrate that the proposed method can (1) achieve performance comparable in self-supervised depth estimation as compared with other state-of-the-art methods when taking clear images as input; (2) predict vivid depth map for images contaminated by various levels of haze when the network trained with previous framework fails; and (3) accurately estimate visibility, airlight, and PM2.5 mass concentrations. Beneficial applications can be developed based on the presented work to contribute to high-precise and dynamic geoinformation reconstruction, transportation, meteorology, and smart city.

Performance Investigation of Generalized Rain Pattern Absorption Attention Network for Single-Image Deraining

Rainy weather conditions are challenging issues for many computer vision applications. Rain streaks and rain patterns are two crucial environmental factors that degrade the visual appearance of high-definition images. A deep attention network-based single-image deraining algorithm is more famous for handling the image with the statistical rain pattern. However, the existing deraining network suffers from the false detection of rain patterns under heavy rain conditions and ineffective detection of directional rain streaks. In this paper, we have addressed the above issues with the following contributions. We propose a multilevel shearlet transform-based image decomposition approach to identify the rain pattern on different scales. The rain streaks in various dimensions are enhanced using a residual recurrent rain feature enhancement module. We adopt the Rain Pattern Absorption Attention Network (RaPaat-Net) to capture and eliminate the rain pattern through the four-dilation factor network. Experiments on synthetic and real-time images demonstrate that the proposed single-image attention network performs better than existing deraining approaches.

Generalized multi-scale image decomposition for new tone manipulation

It is well known that the image multi-scale decomposition plays a foundational role in image processing and a variety of image multi-scale decomposition methods have been derived in various image processing tasks up till now. In many cases, the image decomposition needs to satisfy the conditions: edge-preserving or non-scale-hybridizing. Unfortunately, there has been no such multi-scale image decomposition approach satisfying both the two conditions simultaneously to our best knowledge. In this paper, one novel multi-scale image decomposition approach satisfying both the two conditions is proposed, which achieves different edge-preserving components under different scales. The proposed approach makes full use of well-designed adaptive mean envelope pixels to construct the segmentation envelopes to separate the different scales. Moreover, based on the multi-scale decomposition, an interesting multi-scale tone manipulation scheme and an effective new filtering algorithm for impulse noise are induced. Furthermore, numerical experiments compared with the existent methods are given to demonstrate the state-of-the-art performance of the proposed methods.

A new image decomposition approach using pixel-wise analysis sparsity model

Decomposing an image into two ‘simpler’ layers has been widely used in low-level vision tasks, such as image recovery and enhancement. It is an ill-posed problem since the number of unknowns are larger than the input. In this paper, a two-step strategy is introduced, including task-aware priors estimate and a decomposition model. A pixel-wise analysis sparsity model is proposed to regularize the separation layers, which supposes the transformed image generated with analysis operator is sparse. Unlike regularizing all pixels with one penalty weight, we try to estimate each pixel’s sparsity level with task-aware priors and to achieve pixel-wise sparse penalty. Additionally, one separation layer is regularized with both synthesis sparsity model and pixel-wise analysis sparsity model to exploit their complementary mechanisms. Unlike the analysis one utilizing image local features, the synthesis one exploits an over-complete dictionary and non-local similarity cues to provide flexible prior for regularizing the decomposition results. The proposed model is solved by an alternating optimization algorithm. We evaluate it with two applications, Retinex model and rain streaks removal. Extensive experiments on multiple enhancement datasets, many synthetic and real rainy images demonstrate that our method can remove imaging noise during Retinex decomposition, and can produce high fidelity deraining results. It achieves competing performance in terms of quantitative metrics and visual quality compared with the state-of-the-art methods.

TTCNN: A Breast Cancer Detection and Classification towards Computer-Aided Diagnosis Using Digital Mammography in Early Stages

Breast cancer is a major research area in the medical image analysis field; it is a dangerous disease and a major cause of death among women. Early and accurate diagnosis of breast cancer based on digital mammograms can enhance disease detection accuracy. Medical imagery must be detected, segmented, and classified for computer-aided diagnosis (CAD) systems to help the radiologists for accurate diagnosis of breast lesions. Therefore, an accurate breast cancer detection and classification approach is proposed for screening of mammograms. In this paper, we present a deep learning system that can identify breast cancer in mammogram screening images using an “end-to-end” training strategy that efficiently uses mammography images for computer-aided breast cancer recognition in the early stages. First, the proposed approach implements the modified contrast enhancement method in order to refine the detail of edges from the source mammogram images. Next, the transferable texture convolutional neural network (TTCNN) is presented to enhance the performance of classification and the energy layer is integrated in this work to extract the texture features from the convolutional layer. The proposed approach consists of only three layers of convolution and one energy layer, rather than the pooling layer. In the third stage, we analyzed the performance of TTCNN based on deep features of convolutional neural network models (InceptionResNet-V2, Inception-V3, VGG-16, VGG-19, GoogLeNet, ResNet-18, ResNet-50, and ResNet-101). The deep features are extracted by determining the best layers which enhance the classification accuracy. In the fourth stage, by using the convolutional sparse image decomposition approach, all the extracted feature vectors are fused and, finally, the best features are selected by using the entropy controlled firefly method. The proposed approach employed on DDSM, INbreast, and MIAS datasets and attained the average accuracy of 97.49%. Our proposed transferable texture CNN-based method for classifying screening mammograms has outperformed prior methods. These findings demonstrate that automatic deep learning algorithms can be easily trained to achieve high accuracy in diverse mammography images, and can offer great potential to improve clinical tools to minimize false positive and false negative screening mammography results.

Endoscopy image restoration: A study of the kernel estimation from specular highlights

Endoscopy images show part of the gastrointestinal tract or other parts of the human body. Due to the complex environment in which this tract is located in the human body and the limitations of the image acquisition equipment, endoscopy images may present blur and specular highlights that are common types of degradation. In this paper, we present a blind deblurring approach for endoscopy images that estimate the blur kernel from the fusion of the specular highlights. The proposed method can be divided into two phases. First, the specular highlights are precisely extracted from the original image using a sparse and low-rank image decomposition approach. Then, these highlights are automatically clustered and fused according to the spatial and intensity features. In this way, potential kernels are generated and used for the deblurring process. The experimental results show that the proposed method presents an improved and comparable performance when considered other state-of-the-art methods that use highlights segmentation and image deblurring.

A three stage integrated denoising approach for grey scale images

Restoration of the images contaminated with Additive White Gaussian Noise (AWGN) is a fundamental operation at the interface of statistical and functional analysis. It is an extremely exigent job to design a noise suppression algorithm with edge and feature detail preservation owing to random manifestation of the noise amongst the pixels. In spite of documentation of the sophisticated denoising algorithms in literature, a desired amount of applicability is not achieved as they leave residual noise, create artifacts and remove fine structures. In this paper we propose an integrated image denoising algorithm which exploits the bit plane slicing based image decomposition approach for residual noise removal especially at higher noise levels. The intuitive idea of segregation of noise in decomposed bit planes and selective criterion of denoising lower order bit planes with adaptive bitonic filtering is able to preserve feature details while showing strong denoising performance. Experimental analysis shows that the proposed methodology gives considerable results at low noise values and outperforms other existing techniques at higher noise values both in terms of qualitative and quantitative performance.

A new cartoon–texture image decomposition approach with smoothing spline interpolation

In this paper, we present an effective approach to decompose an image into its cartoon part and texture part without losing edge information. The cartoon and texture parts of an image have their own importance in various applications like medical imaging and image segmentation, etc. Although image decomposition is a widely explored area and a number of approaches are available in the literature, but most of these methods are unable to preserve the edge information effectively. The proposed method is able to deal with this drawback. The proposed method involves two simple steps texture detection followed by smoothing. We demonstrate the experimental results with some state-of-the-art methods to show the efficiency of the proposed approach.

A new computational approach for edge-preserving image decomposition

Decomposition of an image into its cartoon part and texture part has been an interesting area of research. It is an important pre-processing step in many computer vision and image processing techniques such as image segmentation, pattern matching, object recognition, tone mapping as both cartoon and texture parts contain two different kinds of information. Significant work is already available in the literature. At the time of decomposition of an input image into its cartoon and texture part, we required texture-smoothing along with edge-preservation. Most of the approaches available in the literature establish a trade-off between the edge preservation and the texture-smoothing. This common drawback of the existing approaches motivates us to design a new image decomposition method by which we can achieve texture-smoothing without having any loss of edge information in cartoon part. To achieve this aim we introduce a new approach based on Joint bilateral filter and DCT (Discrete cosine transform). We show experimental results on several images to show the effectiveness of our method and comparison with some state-of-the-art methods.

Efficient multithreading for manycore processor: Multidimensional domain decomposition using Intel® TBB

The Insight Toolkit (ITK) utilizes a generic design for image processing filters that allows many developers to rapidly implement new algorithms. While ITK filters benefit from a platform-independent and versatile multithreading capability, the current implementation does not easily achieve high performance. First, ITK relies on a static decomposition of the image into subsets of equal size which is highly inefficient when the computational complexity varies between subsets (unbalanced workloads). Second, the current domain decomposition is limited to subdivide the input domain along a single dimension (typically the slice dimension in a 3-D volume), which causes a multithreading under-utilization when the number of threads is larger than the size of this dimension when using massively parallel compute systems. We previously presented a new itk::TBBImageToImageFilter class that replaced the static task decomposition by a dynamic task decomposition for improved workload balancing, in which the job scheduling task was optimized using the Intel® Threading Building Blocks (TBB) library. In this work, we propose a new multidimensional dynamic image decomposition approach that allows decomposition over an arbitrary number of dimensions. This new generic multithreading capability, combined with the TBB dynamic task scheduler, substantially improves multithreading performance when using massively parallel processors.

Multitemporal SAR Image Decomposition into Strong Scatterers, Background, and Speckle

Speckle phenomenon in synthetic aperture radar (SAR) images makes their visual and automatic interpretation a difficult task. To reduce strong fluctuations due to speckle, total variation (TV) regularization has been proposed by several authors to smooth out noise without blurring edges. A specificity of SAR images is the presence of strong scatterers having a radiometry several orders of magnitude larger than their surrounding region. These scatterers, especially present in urban areas, limit the effectiveness of TV regularization as they break the assumption of an image made of regions of constant radiometry. To overcome this limitation, we propose in this paper an image decomposition approach. There exist numerous methods to decompose an image into several components, notably to separate textural and geometrical information. These decomposition models are generally recast as energy minimization problems involving a different penalty term for each of the components. In this framework, we propose an energy suitable for the decomposition of SAR images into speckle, a smooth background, and strong scatterers, and discuss its minimization using max-flow/min-cut algorithms. We make the connection between the minimization problem considered, involving the L0 pseudonorm, and the generalized likelihood ratio test used in detection theory. The proposed decomposition jointly performs the detection of strong scatterers and the estimation of the background radiometry. Given the increasing availability of time series of SAR images, we consider the decomposition of a whole time series. New change detection methods can be based on the temporal analysis of the components obtained from our decomposition.

An efficient approach for structure-texture image decomposition with edge-preservation

The decomposition of an image into its structural part and texture part is a difficult task and has application in various areas such as medical imaging and image segmentation, etc. Various methods have been proposed to decompose an image into its structure part containing large scale variation in pixel intensity and texture part containing small scale variation in pixel intensity. Unfortunately almost all available methods in the literature have a common drawback that they remove or blur the fine geometric edges while removing the texture, as both geometrical edges and texture both belongs to the high frequency domain. In this work, we propose a new approach to deal with this drawback. The proposed approach effectively remove the texture without affecting the geometrical edges. Also we compare the experimental results with state-of-the-art methods to show the efficiency and effectiveness of proposed approach.

Decomposition and extraction: a new framework for visual classification.

In this paper, we present a novel framework for visual classification based on hierarchical image decomposition and hybrid midlevel feature extraction. Unlike most midlevel feature learning methods, which focus on the process of coding or pooling, we emphasize that the mechanism of image composition also strongly influences the feature extraction. To effectively explore the image content for the feature extraction, we model a multiplicity feature representation mechanism through meaningful hierarchical image decomposition followed by a fusion step. In particularly, we first propose a new hierarchical image decomposition approach in which each image is decomposed into a series of hierarchical semantical components, i.e, the structure and texture images. Then, different feature extraction schemes can be adopted to match the decomposed structure and texture processes in a dissociative manner. Here, two schemes are explored to produce property related feature representations. One is based on a single-stage network over hand-crafted features and the other is based on a multistage network, which can learn features from raw pixels automatically. Finally, those multiple midlevel features are incorporated by solving a multiple kernel learning task. Extensive experiments are conducted on several challenging data sets for visual classification, and experimental results demonstrate the effectiveness of the proposed method.

Intrinsic Image Decomposition Using a Sparse Representation of Reflectance

Intrinsic image decomposition is an important problem that targets the recovery of shading and reflectance components from a single image. While this is an ill-posed problem on its own, we propose a novel approach for intrinsic image decomposition using reflectance sparsity priors that we have developed. Our sparse representation of reflectance is based on a simple observation: Neighboring pixels with similar chromaticities usually have the same reflectance. We formalize and apply this sparsity constraint on local reflectance to construct a data-driven second-generation wavelet representation. We show that the reflectance component of natural images is sparse in this representation. We further propose and formulate a global sparse constraint on reflectance colors using the assumption that each natural image uses a small set of material colors. Using this sparse reflectance representation and the global constraint on a sparse set of reflectance colors, we formulate a constrained $(l_1)$-norm minimization problem for intrinsic image decomposition that can be solved efficiently. Our algorithm can successfully extract intrinsic images from a single image without using color models or any user interaction. Experimental results on a variety of images demonstrate the effectiveness of the proposed technique.

A novel image decomposition approach and its applications

The current state-of-the-art edge-preserving decomposition techniques may not be able to fully separate textures while preserving edges. This may generate artifacts in some applications, e.g., edge detection, texture transfer, etc. To solve this problem, a novel image decomposition approach based on explicit texture separation from large scale components of an image is presented. We first apply a Gaussian structure-texture decomposition, to separate the majority of textures out of the input image. However, residual textures are still visible around the strong edges. To remove these residuals, an asymmetric sampling operator is proposed and followed by a joint bilateral correction to remove an excessive blur effect. We demonstrate that our approach is well suited for the tasks such as texture transfer, edge detection, non-photorealistic rendering, and tone mapping. The results show our approach outperforms existing state-of-the-art image decomposition approaches.

Noise reduction using multiscale bilateral decomposition for digital color images

In this paper, we propose a noise reduction algorithm for digital color images using a nonlinear image decomposition approach. Most existing noise reduction methods do not adequately consider spatial correlation of color noise in digital color images. Color noise components in color images captured by digital cameras are observed as irregular grains with various sizes and shapes, which are spatially randomly distributed. We use a modified multiscale bilateral decomposition to effectively separate signal and mixed-type noise components, in which a noisy input image is decomposed into a base layer and several detail layers. A base layer contains strong edges, and most of noise components are contained in detail layers. Noise components in detail layers are reduced by an adaptive thresholding function. We obtain a denoised image by combining a base layer and noise-reduced detail layers. Experimental results show the effectiveness of the proposed algorithm, in terms of both the peak signal-to-noise ratio and visual quality.