Multiscale Image Decomposition Research Articles

Utilizing data-driven algorithms for blind modal parameter identification of structures from output-only video measurements

In structural dynamics and vibration analysis, modal identification plays a pivotal role in understanding and characterizing the dynamic behavior of complex systems. Traditional modal analysis techniques heavily rely on accurate sensor placement and knowledge of the excitation forces, which may not always be feasible or practical in real-world scenarios. Recently, non-contact video-based modal analysis methods for structures with arbitrary complexity using computer vision techniques have gained much importance. But these methods need to know ahead of time where the structure's natural frequency ranges are, and they use steerable pyramids, which are complicated multi-scale image decomposition filters. To address these issues, a blind modal identification technique is suggested to extract the modal parameters (modal frequency and mode shapes) from the recorded structural vibration video signal. The proposed algorithm for unsupervised machine learning is the Non-Negative Matrix Factorization (NNMF) method, which is combined with the Generalized Complexity Pursuit (GCP) method for blind source separation. The NNMF algorithm is directly applied to the raw pixel-time series made from the video data to get the temporal components, which are then separated using GCP to find the individual modal frequencies and mode shapes. The above algorithm is first validated on a numerical model and then implemented on laboratory scale models (cantilever, single-degree, and multi-degree frame models). Finally, the proposed methodology is implemented on real-world recorded structural vibration videos to determine its noise sensitivity, such as the Tacoma Narrows bridge and the London Millennium footbridge. The modal parameters extracted are compared with those from the available literature for validation. The estimation errors obtained from all the validations are well below 1%, which makes the technique quite suitable and reliable for structural vibration monitoring by identifying and reproducing complex vibration modes.

Malignant melanoma detection using multi-scale image decomposition and a new ensemble-learning scheme

Malignant melanoma detection using multi-scale image decomposition and a new ensemble-learning scheme

Generalized multi-scale image decomposition for new tone manipulation

It is well known that the image multi-scale decomposition plays a foundational role in image processing and a variety of image multi-scale decomposition methods have been derived in various image processing tasks up till now. In many cases, the image decomposition needs to satisfy the conditions: edge-preserving or non-scale-hybridizing. Unfortunately, there has been no such multi-scale image decomposition approach satisfying both the two conditions simultaneously to our best knowledge. In this paper, one novel multi-scale image decomposition approach satisfying both the two conditions is proposed, which achieves different edge-preserving components under different scales. The proposed approach makes full use of well-designed adaptive mean envelope pixels to construct the segmentation envelopes to separate the different scales. Moreover, based on the multi-scale decomposition, an interesting multi-scale tone manipulation scheme and an effective new filtering algorithm for impulse noise are induced. Furthermore, numerical experiments compared with the existent methods are given to demonstrate the state-of-the-art performance of the proposed methods.

A multimodal fusion method for Alzheimer's disease based on DCT convolutional sparse representation.

The medical information contained in magnetic resonance imaging (MRI) and positron emission tomography (PET) has driven the development of intelligent diagnosis of Alzheimer's disease (AD) and multimodal medical imaging. To solve the problems of severe energy loss, low contrast of fused images and spatial inconsistency in the traditional multimodal medical image fusion methods based on sparse representation. A multimodal fusion algorithm for Alzheimer' s disease based on the discrete cosine transform (DCT) convolutional sparse representation is proposed. The algorithm first performs a multi-scale DCT decomposition of the source medical images and uses the sub-images of different scales as training images, respectively. Different sparse coefficients are obtained by optimally solving the sub-dictionaries at different scales using alternating directional multiplication method (ADMM). Secondly, the coefficients of high-frequency and low-frequency subimages are inverse DCTed using an improved L1 parametric rule combined with improved spatial frequency novel sum-modified SF (NMSF) to obtain the final fused images. Through extensive experimental results, we show that our proposed method has good performance in contrast enhancement, texture and contour information retention.

Diffraction Separation and Least-Squares Imaging Based on Multiscale and Multidirectional Wavefield and Image Decomposition

Diffraction Separation and Least-Squares Imaging Based on Multiscale and Multidirectional Wavefield and Image Decomposition

Infrared and visible image fusion algorithm based on spatial domain and image features.

Multi-scale image decomposition is crucial for image fusion, extracting prominent feature textures from infrared and visible light images to obtain clear fused images with more textures. This paper proposes a fusion method of infrared and visible light images based on spatial domain and image features to obtain high-resolution and texture-rich images. First, an efficient hierarchical image clustering algorithm based on superpixel fast pixel clustering directly performs multi-scale decomposition of each source image in the spatial domain and obtains high-frequency, medium-frequency, and low-frequency layers to extract the maximum and minimum values of each source image combined images. Then, using the attribute parameters of each layer as fusion weights, high-definition fusion images are through adaptive feature fusion. Besides, the proposed algorithm performs multi-scale decomposition of the image in the spatial frequency domain to solve the information loss problem caused by the conversion process between the spatial frequency and frequency domains in the traditional extraction of image features in the frequency domain. Eight image quality indicators are compared with other fusion algorithms. Experimental results show that this method outperforms other comparative methods in both subjective and objective measures. Furthermore, the algorithm has high definition and rich textures.

LSTM based texture classification and defect detection in a fabric

Texture classification through deep learning is a science of detecting assumptions from various data through different tools of statistics, machine learning, signal processing and algorithm design. The ultimate aim of textile industries is to produce high quality and defect free fabric to the customers. Traditional methodologies involve manual inspection of every fabric produced which in turn seems to be tedious and time consuming. When Long Short Term Memory (LSTM) is applied to the defect detection and texture classification of a fabric the process leads to high efficiency. Industry can maintain the database which consists of pattern as well as the history of defects present in the fabric. The proposed deep learning technique on LSTM infers the details about the fabric through digital images. The defects in fabric are identified using LSTM method. The defects can be identified irrespective of complex patterns. The obtained images are converted to RGB images and compared with threshold levels for pattern recognition. The obtained factors from proposed technique is pattern of the fabric and location of the defects which include scratches, perforations etc. An unsupervised learning algorithm is proposed to classify the defect percentage present in the fabric and classifying the pattern in the fabric. Here multi-scale curvelet image decomposition and sub band decomposition is used to identify pattern and defects in the fabric. The defect detection involves two different phases. The first phase involves in analyzing defect free images. The second phase incorporated LSTM techniques to identify the defects by using logistic regression. The trained images are decomposed as blocks and the algorithm proposed has the capability to achieve defect detection and pattern recognition in a less computational time.

Green fluorescent protein and phase contrast image fusion via Spectral TV filter-based decomposition

The spectral filter is one of the crucial tools for signal scale and texture analysis. The main function is to decompose a raw image into a series of multi-scale signatures in the spectral domain. For cross-modal fusion, most of the existing approaches naively employ the decomposition of spectral features to achieve image reconstruction. Unfortunately, it does not sufficiently explore the potential cues of different level components in the spatial domain. In this work, we describe an effective decomposition approach using the spectral filter. Specifically, we aim to build a multi-scale image decomposition framework from the spectral domain to the spatial domain, which is called the multi-scale spatial decomposition approach (MSD). Moreover, building upon spectral total variation (TV), we characterize the spectral signatures into various spatial levels with different filters. A fusion scheme of green fluorescent protein and phase contrast image is introduced, leveraging the multi-scale spatial decomposition approach. The source image is first separated by applying MSD into three spatial bands, namely high-frequency bands, low-frequency bands and structure bands. Our method is simple and effective. The divided bands are obtained by a set of filters and could be processed with the specific fusion strategies to extract the features of the source image. In addition, experiments show that our method produces promising results than other current popular methods.

Denoising Seismic Data via a Threshold Shrink Method in the Non-Subsampled Contourlet Transform Domain

In seismic exploration, effective seismic signals can be seriously distorted by and interfered with noise, and the performance of traditional seismic denoising approaches can hardly meet the requirements of high-precision seismic exploration. To remarkably enhance signal-to-noise ratios (SNR) and adapt to high-precision seismic exploration, this work exploits the non-subsampled contourlet transform (NSCT) and threshold shrink method to design a new approach for suppressing seismic random noise. NSCT is an excellent multiscale, multidirectional, and shift-invariant image decomposition scheme, which can not only calculate exact contourlet transform coefficients through multiresolution analysis but also give an almost optimized approximation. It has better high-frequency response and stronger ability to describe curves and surfaces. Specifically, we propose to utilize the superior performance NSCT to decomposing the noisy seismic data into various frequency sub-bands and orientation response sub-bands, obtaining fine enough transform high frequencies to effectively achieve the separation of signals and noises. Besides, we use the adaptive Bayesian threshold shrink method instead of traditional handcraft threshold scheme for denoising the high-frequency sub-bands of NSCT coefficients, which pays more attention to the internal characteristics of the signals/data itself and improve the robustness of method, which can work better for preserving richer structure details of effective signals. The proposed method can achieve seismic random noise attenuation while retaining effective signals to the maximum degree. Experimental results reveal that the proposed method is superior to wavelet-based and curvelet-based threshold denoising methods, which increases synthetic seismic data with lower SNR from −8.2293 dB to 8.6838 dB, and 11.8084 dB and 9.1072 dB higher than two classic sparse transform based methods, respectively. Furthermore, we also apply the proposed method to process field data, which achieves satisfactory results.

Image Fusion and Stylization Processing Based on Multiscale Transformation and Convolutional Neural Network.

With the continuous development of imaging sensors, images contain more and more information, the images presented by different types of sensors are different, and the images obtained by the same type of sensors under different parameters or conditions are also different. Multisource image fusion technology combines images acquired by different types of sensors or the same type of sensors with different parameter settings, which makes the image information more complete, compensates for the limitations of images of the same type, and also allows you to save information about the characteristics of the original image. Multimodal image mosaic and multifocal image mosaic have been studied in detail in two directions. On the one hand, a method based on frequency domain transformation is used for multiscale image decomposition. On the other hand, image extraction with neural network-based methods is proposed. The technology of convolutional neural networks (CNNs) allows to extract richer texture features. However, when using this method for fusion, it is difficult to obtain an accurate decision map, and there are artifacts in the fusion boundary. Based on this, a multifocal fusion method based on a two-stage CNN is proposed. Train the advanced intensive network to classify input image blocks as focus, and then use the appropriate merge rules to get the ideal decision tree. In addition, several versions of the fuzzy learning set have been developed to improve network performance. Experimental results show that the frames of the first stage proposed by the algorithm make it possible to obtain an accurate decision scheme and that the frames of the second stage make it possible to eliminate the pseudo-shadow of the integration boundary.

Generalized Multi-Scale Image Decomposition for New Tone Manipulation

Generalized Multi-Scale Image Decomposition for New Tone Manipulation

Multiscale decomposition and fusion‐based color contrast restoration for various water‐colored environments

AbstractUnderwater image processing is a vast area of research to identify blurred objects in the ocean or sea environments. The light absorbed and scattered by the underwater medium create artifacts in the acquired images due to low‐light effects, poor visibility, darkness, illumination variation, and color contrast degradation. In this research work, a novel optical image preprocessing framework was proposed to restore the color contrast of images obtained from various water‐colored environments. Domain transform‐based multiscale image decomposition method was used to extract the base, residual, and detail layers. Then, the smoothed image was reconstructed from the base layer by applying the adaptive iterative backward‐projection algorithm. To obtain dehazed images, a multiscale fusion‐based algorithm was developed, which fused the detail layers and reconstructed the images. The outcome of the proposed framework revealed that the contrast degradation and low‐light‐related artifacts present in the acquired images from various water‐colored environments were found to be considerably reduced. The experimental results of the proposed method exhibit scalable improvements over the other methods in terms of visual perception and estimated performance metrics (underwater image quality measure = 6.1186; peak signal‐to‐noise ratio = 26.68; feature points matching = 96 points; semantic image segmentation accuracy = 0.8465 and processing time of 0.11 seconds on a single image). The novelty of this method relies on the fusion of well‐established techniques, which outperformed the advanced techniques like deep learning method in terms of simplicity, speed, performance, datasets, image resolution, and applications.

Non-linear and selective fusion of cross-modal images

Existing image fusion methods pay little research attention to human visual characteristics. However, human visual characteristics play an important role in visual processing tasks. To solve this problem, we propose a cross-modal image fusion method that combines illuminance factors and attention mechanisms. Human visual characteristics are studied and simulated in cross-modal image fusion task. Firstly, in order to reject high and low-frequency mixing and reduce the halo effect, we perform cross-modal image multi-scale decomposition. Secondly, in order to remove highlights, the visual saliency map and the deep feature map are combined with the illuminance fusion factor to perform high-low frequency non-linear fusion. Thirdly, the feature maps are selected through a channel attention network to obtain the final fusion map. Finally, we validate our image fusion method on public datasets of infrared and visible images. The experimental results demonstrate the superiority of our fusion method under the complex illumination environment. In addition, the experimental results also demonstrate the effectiveness of our simulation of human visual characteristics.

Early Detection and Classification Approach for Plant Diseases based on MultiScale Image Decomposition

This paper presents a new and powerful approach for detecting and classifying leaf diseases for plant diagnosis with high accuracy. The main contribution of this paper is that a hybrid approach is proposed by using the combination of Partial Differential Equations (PDE) based image decomposition, segmentation, feature extraction, features selection and classification aiming to improve the classification accuracy and provide an excellent diagnosis. The TV-L1 Total variation model is adopted to separate the original image into texture and object components. Segmentation will be done only on the object component. Then texture, color, vein and shape features are extracted and merged in a feature vector using the codebook method. Moreover, features are selected by the RelieF feature selection algorithm to keep only relevant ones. In the classification, only selected features will be used and passed to the Multiclass Support Vector Machine algorithm SVM. The proposed approach is implemented and tested on the PV Plant Village dataset and provided a good and greater classification accuracy compared with the existing approaches from the literature. The obtained results proved that the use of PDE influences on the segmentation, which in turn, allowed us to identify correctly the leaves and provide new and optimal features, those features improves the classification accuracy rate to 95.9%.

Deep Residual Transform for Multi-scale Image Decomposition

Multi-scale image decomposition (MID) is a fundamental task in computer vision and image processing that involves the transformation of an image into a hierarchical representation comprising of different levels of visual granularity from coarse structures to fine details. A well-engineered MID disentangles the image signal into meaningful components which can be used in a variety of applications such as image denoising, image compression, and object classification. Traditional MID approaches such as wavelet transforms tackle the problem through carefully designed basis functions under rigid decomposition structure assumptions. However, as the information distribution varies from one type of image content to another, rigid decomposition assumptions lead to inefficiently representation, i.e., some scales can contain little to no information. To address this issue, we present Deep Residual Transform (DRT), a data-driven MID strategy where the input signal is transformed into a hierarchy of non-linear representations at different scales, with each representation being independently learned as the representational residual of previous scales at a user-controlled detail level. As such, the proposed DRT progressively disentangles scale information from the original signal by sequentially learning residual representations. The decomposition flexibility of this approach allows for highly tailored representations cater to specific types of image content, and results in greater representational efficiency and compactness. In this study, we realize the proposed transform by leveraging a hierarchy of sequentially trained autoencoders. To explore the efficacy of the proposed DRT, we leverage two datasets comprising of very different types of image content: 1) CelebFaces and 2) Cityscapes. Experimental results show that the proposed DRT achieved highly efficient information decomposition on both datasets amid their very different visual granularity characteristics.

An Efficient Algorithm to Highlight Details in Infrared and Visible Image Fusion

To improve the fusion quality of infrared and visible images and highlight target and scene details, in this paper, a novel infrared and visible image fusion algorithm is proposed. First, a method for combining dynamic range compression and contrast restoration based on a guided filter is adopted to enhance the contrast of visible source images. Second, guided filter-based image multiscale decomposition is used to decompose images into base layers and detail layers. For base layer fusion, a fusion strategy based on the detail and energy measurements of the source image is proposed to determine the pixel value of the fused image base layer such that the energy loss of the fusion can be reduced and the texture detail features are highlighted to obtain more source image details. Finally, recursive separation and weighted histogram equalization methods are applied to optimize the fused image. Experimental results show that the fusion algorithm and fusion strategy proposed in this paper can effectively improve fusion image clarity, while more detailed target and scene information can still be retained.

Research on real-time data transmission and multi-scale video image decomposition of embedded optical sensor array based on machine learning

Aiming at the research of real-time data transmission and multi-scale image decomposition of embedded optical sensor array, the principle, method and fusion strategy of multi-sensor image fusion are studied comprehensively, thoroughly and systematically by combining the imaging characteristics of source image with multi-scale geometric analysis tools using machine learning algorithm. A new quality scalable video image coding framework is also proposed in this paper, which is implemented by a multi-scale online dictionary learning algorithm based on structured sparse video signals. For the purpose of different types of images and image fusion, a new high quality scalable video image coding framework based on machine learning algorithm is proposed on the basis of comprehensive analysis of prior information such as imaging mechanism of image sensor and imaging characteristics of source image. A multi-scale online dictionary learning algorithm based on machine learning for sparse video signal structure is proposed. Through the hierarchical structure of wavelet decomposition, the searching domain of online learning is optimized to a hierarchical sparse block, and its sparse representation coefficients are obtained by using machine learning sparse coding idea. The real-time data transmission of embedded optical sensor array based on machine learning and multi-scale image decomposition algorithm proposed in this paper have good fusion performance, which is of great significance for further research and engineering application of image fusion technology.

Urban change detection analysis utilizing multiresolution texture features from polarimetric SAR images

This paper presents a multiresolution textural approach to change detection in multitemporal polarimetric synthetic aperture radar images. Several change detection methods utilizing wavelet based features from remotely sensed images have been developed previously. The aim of this paper is to investigate and propose a texture based change detection method that applies curvelet and contourlet transforms on polarimetric synthetic aperture radar (SAR) images. The proposed approach exploits curvelet and contourlet based multiscale decomposition of original images where textural information is extracted at various scales and in different directions in terms of statistical moments and energy to generate feature maps. The L1-norm is used in the proposed method to generate the difference image, which is thresholded using the maximum entropy principle to obtain the final change detection map. The results are compared with the changes detected by the wavelet based textural features, plain texture difference, image difference and log ratio methods. Finally, accuracy assessment and comparative analysis are performed for change maps carried out to review missed changes, false-alarms and the overall accuracy of this method. The results are also analysed statistically with area under ROC curve (AUC). The study found that the proposed method produces results with high change detection accuracy with better edge continuity and greater AUC.

Multiscale empirical mode decomposition of density fluctuation images very near above and below the critical point of S[formula omitted

We use a multiscale approach to investigate the dynamics of fluctuations near the critical point of sulfur hexafluoride (SF6) in microgravity. Rather than increasing the fitting model’s complexity during the critical temperature crossing, we used a different approach to finding the thermal diffusivity coefficient (above critical temperature), which can then be distinguished from an effective diffusion coefficient (below critical temperature). We first separate different spatial scales from the original images using the Bidimensional Empirical Mode Decomposition (BEMD) technique. The spatial scale represented by an Intrinsic Mode Function (IMF) image was analyzed using the Dynamic Differential Method (DDM). The Intermediate Scattering Function (ISF) of each IMF was used for computing the structure factor and the relaxation time of fluctuations. We found that the first IMF returns over 90% of the spatial and temporal knowledge contained in the original image, providing thus thermal diffusivity coefficient above the critical temperature and effective diffusion coefficients below the critical temperature very close in magnitude. The relaxation time associated with the distinguishable structures observed in the second IMF could be attributed to the fractal nature of fluctuations. and to light scattering at low wavenumber during the stationary behavior and the transient evolution of the critical fluid cell, which are not easy to detect in the original image. The third order IMF presents no noticeable structure, and the associated relaxation time is not physically significant.

Light field all-in-focus image fusion based on spatially-guided angular information

Compared to traditional 2D images, light field images record both spatial and angular information of the scene, which can provide more data for image fusion. In this paper, a light field all-in-focus image fusion algorithm based on spatially-guided angular information is proposed. In the proposed method, the initial weight maps carrying the angular information are calculated by comparing the block variance of the 4D light field data. The initial weight maps are then guided by digital refocused images carrying the spatial information to obtain the refined weight maps. In the refocused image multi-scale decomposition, the micro-lens calibration error is considered and the additional edge layers are extracted to suppress the edge artifacts. Experiments demonstrate the effectiveness of the proposed algorithm. Quantitative evaluation results show that the proposed algorithm performs the best in the feature-based index and structural similarity-based index without sacrificing the information and perceptual sharpness of the fused image.