Structured Sparse Representation Research Articles

Learning the Hybrid Nonlocal Self-Similarity Prior for Image Restoration

As an immensely important characteristic of natural images, the nonlocal self-similarity (NSS) prior has demonstrated great promise in a variety of inverse problems. Unfortunately, most current methods utilize either the internal or the external NSS prior learned from the degraded image or training images. The former is inevitably disturbed by degradation, while the latter is not adapted to the image to be restored. To mitigate such problems, this work proposes to learn a hybrid NSS prior from both internal images and external training images and employs it in image restoration tasks. To achieve our aims, we first learn internal and external NSS priors from the measured image and high-quality image sets, respectively. Then, with the learned priors, an efficient method, involving only singular value decomposition (SVD) and a simple weighting method, is developed to learn the HNSS prior for patch groups. Subsequently, taking the learned HNSS prior as the dictionary, we formulate a structural sparse representation model with adaptive regularization parameters called HNSS-SSR for image restoration, and a general and efficient image restoration algorithm is developed via an alternating minimization strategy. The experimental results indicate that the proposed HNSS-SSR-based restoration method exceeds many existing competition algorithms in PSNR and SSIM values.

Image restoration via joint low-rank and external nonlocal self-similarity prior

Recent studies have revealed that joint priors, such as joint sparsity and external nonlocal self-similarity (ENSS) prior and joint low-rank and sparsity prior, are extremely effective in various image inverse problems. Few works, however, make use of both low-rank and ENSS priors. With this in mind, in this paper we propose a new joint prior, namely LRENSS prior, which utilizes low-rank and ENSS priors jointly in a unified framework, and successfully adapt the proposed LRENSS prior to image restoration problems. Specifically, low-rank and ENSS priors are bridged by treating ENSS prior as dictionaries for structural sparse representation. Further, an elegant block coordinate descent method is developed to solve the corresponding optimization problem. The proposed LRENSS prior is validated on image denoising and image deblurring tasks. Experimental results illustrate that the proposed LRENSS prior has better performance than other state-of-the-art algorithms in both qualitative and quantitative assessments.

Multi-layer clustering-based residual sparsifying transform for low-dose CT image reconstruction.

The recently proposed sparsifying transform (ST) models incur low computational cost and have been applied to medical imaging. Meanwhile, deep models with nested network structure reveal great potential for learning features in different layers. In this study, we propose a network-structured ST learning approach for X-ray computed tomography (CT), which we refer to as multi-layer clustering-based residual sparsifying transform (MCST) learning. The proposed MCST scheme learns multiple different unitary transforms in each layer by dividing each layer's input into several classes. We apply the MCST model to low-dose CT (LDCT) reconstruction by deploying the learned MCST model into the regularizer in penalized weighted least squares (PWLS) reconstruction. The proposed MCST model combines a multi-layer sparse representation structure with multiple clusters for the features in each layer that are modeled by a rich collection of transforms. We train the MCST model in an unsupervised manner via a block coordinate descent (BCD) algorithm. Since our method is patch-based, the training can be performed with a limited set of images. For CT image reconstruction, we devise a novel algorithm called PWLS-MCST by integrating the pre-learned MCST signal model with PWLSoptimization. We conducted LDCT reconstruction experiments on XCAT phantom data, Numerical Mayo Clinical CT dataset and "LDCT image and projection dataset" (Clinical LDCT dataset). We trained the MCST model with two (or three) layers and with five clusters in each layer. The learned transforms in the same layer showed rich features while additional information is extracted from representation residuals. Our simulation results and clinical results demonstrate that PWLS-MCST achieves better image reconstruction quality than the conventional filtered back-projection (FBP) method and PWLS with edge-preserving (EP) regularizer. It also outperformed recent advanced methods like PWLS with a learned multi-layer residual sparsifying transform (MARS) prior and PWLS with a union of learned transforms (ULTRA), especially for displaying clear edges and preserving subtledetails. In this work, a multi-layer sparse signal model with a nested network structure is proposed. We refer this novel model as the MCST model that exploits multi-layer residual maps to sparsify the underlying image and clusters the inputs in each layer for accurate sparsification. We presented a new PWLS framework with a learned MCST regularizer for LDCT reconstruction. Experimental results show that the proposed PWLS-MCST provides clearer reconstructions than several baseline methods. The code for PWLS-MCST is released at https://github.com/Xikai97/PWLS-MCST.

Joint group dictionary-based structural sparse representation for image restoration

With the aid of image nonlocal self-similarity (NSS), recent studies have indicated that structural sparse representation (SSR) models with patch groups as processing units have a great potential in various image inverse problems. Most of existing works, however, only use the dictionary learned from internal or external patch groups for sparse representation. Consequently, such methods generally overfit degradation or do not adapt to the given image. In this paper, we propose a novel sparse representation model, termed joint group dictionary-based SSR (JGD-SSR) model, to alleviate the above problem. Compared with traditional SSR models, the proposed JGD-SSR provides an effective mechanism to exploit internal and external dictionaries jointly. Moreover, to make the proposed model more robust, the connection between JGD-SSR and maximum a posteriori (MAP) estimation is revealed to adaptively estimate regularization parameters. When we apply the proposed JGD-SSR to image restoration tasks, an alternating minimization scheme is developed to solve the resulting optimization problems. The experimental results, including image denoising, deblocking, and deblurring, demonstrate that the proposed JGD-SSR-based restoration algorithm outperforms many state-of-the-art competing methods in terms of both objective and visual qualities.

Remote Heart Rate Estimation by Pulse Signal Reconstruction Based on Structural Sparse Representation

In recent years, the physiological measurement based on remote photoplethysmography has attracted wide attention, especially since the epidemic of COVID-19. Many researchers paid great efforts to improve the robustness of illumination and motion variation. Most of the existing methods divided the ROIs into many sub-regions and extracted the heart rate separately, while ignoring the fact that the heart rates from different sub-regions are consistent. To address this problem, in this work, we propose a structural sparse representation method to reconstruct the pulse signals (SSR2RPS) from different sub-regions and estimate the heart rate. The structural sparse representation (SSR) method considers that the chrominance signals from different sub-regions should have a similar sparse representation on the combined dictionary. Specifically, we firstly eliminate the signal deviation trend using the adaptive iteratively re-weighted penalized least squares (Airpls) for each sub-region. Then, we conduct the sparse representation on the combined dictionary, which is constructed considering the pulsatility and periodicity of the heart rate. Finally, we obtain the reconstructed pulse signals from different sub-regions and estimate the heart rate with a power spectrum analysis. The experimental results on the public UBFC and COHFACE datasets demonstrate the significant improvement for the accuracy of the heart rate estimation under realistic conditions.

Nonconvex Structural Sparsity Residual Constraint for Image Restoration.

This article proposes a novel nonconvex structural sparsity residual constraint (NSSRC) model for image restoration, which integrates structural sparse representation (SSR) with nonconvex sparsity residual constraint (NC-SRC). Although SSR itself is powerful for image restoration by combining the local sparsity and nonlocal self-similarity in natural images, in this work, we explicitly incorporate the novel NC-SRC prior into SSR. Our proposed approach provides more effective sparse modeling for natural images by applying a more flexible sparse representation scheme, leading to high-quality restored images. Moreover, an alternating minimizing framework is developed to solve the proposed NSSRC-based image restoration problems. Extensive experimental results on image denoising and image deblocking validate that the proposed NSSRC achieves better results than many popular or state-of-the-art methods over several publicly available datasets.

A Hybrid Structural Sparsification Error Model for Image Restoration.

Recent works on structural sparse representation (SSR), which exploit image nonlocal self-similarity (NSS) prior by grouping similar patches for processing, have demonstrated promising performance in various image restoration applications. However, conventional SSR-based image restoration methods directly fit the dictionaries or transforms to the internal (corrupted) image data. The trained internal models inevitably suffer from overfitting to data corruption, thus generating the degraded restoration results. In this article, we propose a novel hybrid structural sparsification error (HSSE) model for image restoration, which jointly exploits image NSS prior using both the internal and external image data that provide complementary information. Furthermore, we propose a general image restoration scheme based on the HSSE model, and an alternating minimization algorithm for a range of image restoration applications, including image inpainting, image compressive sensing and image deblocking. Extensive experiments are conducted to demonstrate that the proposed HSSE-based scheme outperforms many popular or state-of-the-art image restoration methods in terms of both objective metrics and visual perception.

Multi‐Traffic Targets Tracking Based on an Improved Structural Sparse Representation with Spatial‐Temporal Constraint

Vehicles or pedestrians tracking is an important task in intelligent transportation system. In this paper, we propose an online multi-object tracking for intelligent traffic platform that employs improved sparse representation and structural constraint. We first build the spatial-temporal constraint via the geometric relations and appearance of tracked objects, then we construct a robust appearance model by incorporating the discriminative sparse representation with weight constraint and local sparse appearance with occlusion analysis. Finally, we complete data association by using maximum a posteriori in a Bayesian framework in the pursuit for the optimal detection estimation. Experimental results in two challenging vehicle tracking benchmark datasets show that the proposed method has a good tracking performance.

Archetypal Analysis and Structured Sparse Representation for Hyperspectral Anomaly Detection

Hyperspectral images (HSIs) often contain pixels with mixed spectra, which makes it difficult to accurately separate the background signal from the anomaly target signal. To mitigate this problem, we present a method that applies spectral unmixing and structure sparse representation to accurately extract the pure background features and to establish a structured sparse representation model at a sub-pixel level by using the Archetypal Analysis (AA) scheme. Specifically, spectral unmixing with AA is used to unmix the spectral data to obtain representative background endmember signatures. Moreover the unmixing reconstruction error is utilized for the identification of the target. Structured sparse representation is also adopted for anomaly target detection by using the background endmember features from AA unmixing. Moreover, both the AA unmixing reconstruction error and the structured sparse representation reconstruction error are integrated together to enhance the anomaly target detection performance. The proposed method exploits background features at a sub-pixel level to improve the accuracy of anomaly target detection. Comparative experiments and analysis on public hyperspectral datasets show that the proposed algorithm potentially surpasses all the counterpart methods in anomaly target detection.

A multi-objective opposition-based barnacles mating optimization for image super resolution using hyper-Spectral images

Purpose The problems of Super resolution are broadly discussed in diverse fields. Rather than the progression toward the super resolution models for real-time images, operating hyperspectral images still remains a challenging problem. Design/methodology/approach This paper aims to develop the enhanced image super-resolution model using “optimized Non-negative Structured Sparse Representation (NSSR), Adaptive Discrete Wavelet Transform (ADWT), and Optimized Deep Convolutional Neural Network”. Once after converting the HR images into LR images, the NSSR images are generated by the optimized NSSR. Then the ADWT is used for generating the subbands of both NSSR and HRSB images. The residual image with this information is obtained by the optimized Deep CNN. All the improvements on the algorithms are done by the Opposition-based Barnacles Mating Optimization (O-BMO), with the objective of attaining the multi-objective function concerning the “Peak Signal-to-Noise Ratio (PSNR), and Structural similarity (SSIM) index”. Extensive analysis on benchmark hyperspectral image datasets shows that the proposed model achieves superior performance over typical other existing super-resolution models. Findings From the analysis, the overall analysis of the suggested and the conventional super resolution models relies that the PSNR of the improved O-BMO-(NSSR+DWT+CNN) was 38.8% better than bicubic, 11% better than NSSR, 16.7% better than DWT+CNN, 1.3% better than NSSR+DWT+CNN, and 0.5% better than NSSR+FF-SHO-(DWT+CNN). Hence, it has been confirmed that the developed O-BMO-(NSSR+DWT+CNN) is performing well in converting LR images to HR images. Originality/value This paper adopts a latest optimization algorithm called O-BMO with optimized Non-negative Structured Sparse Representation (NSSR), Adaptive Discrete Wavelet Transform (ADWT) and Optimized Deep Convolutional Neural Network for developing the enhanced image super-resolution model. This is the first work that uses O-BMO-based Deep CNN for image super-resolution model enhancement.

Low-Velocity Impact Localization on a Honeycomb Sandwich Panel Using a Balanced Projective Dictionary Pair Learning Classifier.

Carbon-fiber aluminum honeycomb sandwich panels are vulnerable to low-velocity impacts, which can cause structural damage and failures that reduce the bearing performance and reliability of the structure. Therefore, a method for locating such impacts through a sensor network is very important for structural health monitoring. Unlike composite laminates, the stress wave generated by an impact is damped rapidly in a sandwich panel, meaning that the signal qualities measured by different sensors vary greatly, thereby making it difficult to locate the impact. This paper presents a method for locating impacts on carbon-fiber aluminum honeycomb sandwich panels utilizing fiber Bragg grating sensors. This method is based on a projective dictionary pair learning algorithm and uses structural sparse representation for impact localization. The measurement area is divided into several sub-areas, and a corresponding dictionary is trained separately for each sub-area. For each dictionary, the sensors are grouped into main sensors within the sub-area and auxiliary sensors outside the sub-area. A balancing weight factor is added to optimize the proportion of the two types of sensor in the recognition model, and the algorithm for determining the balancing weight factor is designed to suppress the negative effects on the positioning of the sensors with poor signal quality. The experimental results show that on a 300 mm × 300 mm × 15 mm sandwich panel, the impact positioning accuracy of this method is 96.7% and the average positioning error is 0.85 mm, which are both sufficient for structural health monitoring.

Adaptive Nonnegative Sparse Representation for Hyperspectral Image Super-Resolution

As the hyperspectral images (HSIs) usually have a low spatial resolution, HSI super-resolution has recently attracted more and more attention to enhance the spatial resolution of HSIs. A common method is to fuse the low-resolution (LR) HSI with a multispectral image (MSI) whose spatial resolution is higher than the HSI. In this article, we proposed a novel adaptive nonnegative sparse representation-based model to fuse an HSI and its corresponding MSI. First, basing the linear spectral unmixing, the nonnegative structured sparse representation model estimates the sparse codes of the desired high-resolution HSI from both the LR-HSI and the MSI. Then, the adaptive sparse representation can balance the relationship between the sparsity and collaboration by generating a suitable coefficient. Finally, in order to obtain more accurate results, we alternately optimize the spectral basis and coefficients rather than keeping the spectral basis fixed. The alternating direction method of multipliers is applied to solve the proposed optimization problem. The experimental results on both ground-based HSIs and real remote sensing HSIs show the superiority of our proposed approach to some other state-of-the-art HSI super-resolution methods.

Contribution of enhanced DWT and non-negative structured sparse representation for image super resolution: A performance analysis

The image super-resolution methods with deep learning using Convolutional Neural Network (CNN) have been producing admirable advancements. The proposed image resolution model involves the following two main analyses: (i) analysis using Adaptive Discrete Wavelet Transform (ADWT) with Deep CNN and (ii) analysis using Non-negative Structured Sparse Representation (NSSR). The technique termed as NSSR is used to recover the high-resolution (HR) images from the low-resolution (LR) images. The experimental evaluation involves two phases: Training and Testing. In the training phase, the information regarding the residual images of the dataset are trained using the optimized Deep CNN. On the other hand, the testing phase helps to generate the super resolution image using the HR wavelet subbands (HRSB) and residual images. As the main novelty, the filter coefficients of DWT are optimized by the hybrid Fire Fly-based Spotted Hyena Optimization (FF-SHO) to develop ADWT. Finally, a valuable performance evaluation on the two benchmark hyperspectral image datasets confirms the effectiveness of the proposed model over the existing algorithms.

Structural sparse representation with class-specific dictionary for ECG biometric recognition

For traditional person recognition, most methods are knowledge-based and token-based such as the methods using password, ID number or other secret information, and to enhance its security, biometric recognition is widely used in many areas. As a biometric trait, electrocardiogram (ECG) signal is gaining more attention due to the fact that it is not easy to be faked compared with other traits and only exists in living subjects. However, ECG signals are influenced by numerous factors easily and have low stability, resulting in unsatisfied recognition performance. In this paper, we propose a structural sparse representation algorithm for ECG signals and learn class-specific dictionary for each class to address these problems. Firstly, we segment heartbeats for each subject’s signal. Moreover, the mixed regularization l2,1 norm is utilized for ECG heartbeats to obtain a robust and stable feature. It is encouraged that if the atom is used to describe the heartbeat of one subject’s heartbeat, it may be used to describe other heartbeats of the same subject. In addition, a graph regularization is used to minimize the difference between heartbeats of intra-class. Finally, we conduct experiments on three public ECG databases to verify the proposed method and the subject recognition accuracy can reach to 99.66% on PTB database. From the experimental result, the proposed method can take full advantage of ECG structural characteristic and get a better performance compared with state-of-the-art methods.

Image Restoration via Simultaneous Nonlocal Self-Similarity Priors.

Through exploiting the image nonlocal self-similarity (NSS) prior by clustering similar patches to construct patch groups, recent studies have revealed that structural sparse representation (SSR) models can achieve promising performance in various image restoration tasks. However, most existing SSR methods only exploit the NSS prior from the input degraded (internal) image, and few methods utilize the NSS prior from external clean image corpus; how to jointly exploit the NSS priors of internal image and external clean image corpus is still an open problem. In this paper, we propose a novel approach for image restoration by simultaneously considering internal and external nonlocal self-similarity (SNSS) priors that offer mutually complementary information. Specifically, we first group nonlocal similar patches from images of a training corpus. Then a group-based Gaussian mixture model (GMM) learning algorithm is applied to learn an external NSS prior. We exploit the SSR model by integrating the NSS priors of both internal and external image data. An alternating minimization with an adaptive parameter adjusting strategy is developed to solve the proposed SNSS-based image restoration problems, which makes the entire algorithm more stable and practical. Experimental results on three image restoration applications, namely image denoising, deblocking and deblurring, demonstrate that the proposed SNSS produces superior results compared to many popular or state-of-the-art methods in both objective and perceptual quality measurements.

Small Sample Set Inshore Ship Detection From VHR Optical Remote Sensing Images Based on Structured Sparse Representation

Inshore ship detection from very high resolution (VHR) optical remote sensing images has been playing a critical role in various civil and military applications. However, it brings up an important challenge, which is difficult to complete effective and robust feature extraction when valid inshore ship training sample acquired is limited, and the severe imbalance problem exists of positive and negative samples. In order to tackle the abovementioned difficulties, the structured sparse representation model (SSRM) is proposed to achieve inshore ship detection in more effectively and robustly way by circumstances of the small sample set. Here, SSRM has two steps that include inshore ship region proposal (RP) and orientation prediction (OP). Related to the RP process, the error matrix embedded in SSRM not only prevents to build the high-dimension background subdictionary and imbalance problem of positive and negative samples, but also achieves an effective intraclass robustness description of inshore ships and background. For the OP stage, the low-rank constraint of common sharing atoms in SSRM can make inshore ship direction be extracted by their sparse coding. In addition, based on RP and OP guidance, the proposed comprehensive structure voting can achieve an accurate contour detection of inshore ships. Finally, several experimental results employ that Google Earth service, HRSC 2016, and DOTA datasets proved the effectiveness of the proposed method. The results show that proposed inshore ship detection method can provide approximately 83.7% Recall and 72.3% Precision by using only over 100 positive training samples, which outperforms the state of the art methods.

Target Tracking Method Based on Adaptive Structured Sparse Representation With Attention

Considering the problems of motion blur, partial occlusion and fast motion in target tracking, a target tracking method based on adaptive structured sparse representation with attention is proposed. Under the framework of particle filtering, the performance of high-quality templates is enhanced through an attention mechanism. Structure sparseness is used to build candidate target sets and sparse models between candidate samples and local patches of target templates. Combined with the sparse residual method, reconstruction error is reduced. After optimally solving the model, the particle with the highest similarity is selected as the prediction target. The most appropriate scale is selected according to the multiscale factor method. Experiments show that the proposed algorithm has a strong performance when dealing with motion blur, fast motion, partial occlusion.

A Massive Multi-Modal Perception Data Classification Method Using Deep Learning Based on Internet of Things

In recent years, the Internet of Things (IoT) has developed rapidly due to its broad application prospects, and the types of sensing devices are becoming more and more abundant. In many applications, multi-dimensional attributes of monitoring objects are measured by deploying multiple independent heterogeneous data sources, thus heterogeneous multi-source multi-modal sensing data can be obtained. In this paper, we measure the data quality of multi-source and multi-modal data and make full use of the data quality information, a classification and detection method for heterogeneous multi-source and multi-modal sensing data was proposed. The purpose is to ensure the data quality and select some data sources for data transmission in order to save network resources as much as possible. Firstly, the problem caused by heterogeneous multi-modal data is solved by the feature transformation of deep neural network, and an innovative training method is used to extract the multi-modal shared feature expression with strong discriminant ability and low-dimensional characteristics from the original multi-modal high-dimensional data. Then, an algorithm combining deep learning with structural sparse multi-modal feature representation and mode selection is proposed, which innovatively uses deep learning to transform multi-modal data into modal-independent Abstract expressions. Finally, the structural sparse method is used to further select the feature dimension in the abstract expression to reduce the dimension of the final feature. Experiments show that the average false alarm rate of the proposed model is 2.2%, which is obviously superior to other similar algorithms.

Super-resolution reconstruction of neonatal brain magnetic resonance images via residual structured sparse representation.

Magnetic resonance images of neonates, compared with toddlers, exhibit lower signal-to-noise ratio and spatial resolution. In this paper, we propose a novel method for super-resolution reconstruction of neonate images with the help of toddler images, using residual-structured sparse representation with convex regularization. Specifically, we introduce a two-layer image representation, consisting of a base layer and a detail layer, to cater to signal variation across scanners and sites. The base layer consists of the smoothed version of the image obtained via Gaussian filtering. The detail layer is the difference between the original image and the base layer. High-frequency details in the detail layer are borrowed across subjects for super-resolution reconstruction. Experimental results on T1 and T2 images demonstrate that the proposed algorithm can recover fine anatomical structures, and generally outperform the state-of-the-art methods both qualitatively and quantitatively.

Robust Visual Tracking via Patch Descriptor and Structural Local Sparse Representation

Appearance models play an important role in visual tracking. Effective modeling of the appearance of tracked objects is still a challenging problem because of object appearance changes caused by factors, such as partial occlusion, illumination variation and deformation, etc. In this paper, we propose a tracking method based on the patch descriptor and the structural local sparse representation. In our method, the object is firstly divided into multiple non-overlapped patches, and the patch sparse coefficients are obtained by structural local sparse representation. Secondly, each patch is further decomposed into several sub-patches. The patch descriptors are defined as the proportion of sub-patches, of which the reconstruction error is less than the given threshold. Finally, the appearance of an object is modeled by the patch descriptors and the patch sparse coefficients. Furthermore, in order to adapt to appearance changes of an object and alleviate the model drift, an outlier-aware template update scheme is introduced. Experimental results on a large benchmark dataset demonstrate the effectiveness of the proposed method.