Semi-coupled Dictionary Learning Research Articles

CROSS STYLE IMAGE SYNTHESIS THROUGH SEMI COUPLED DICTIONARY LEARNING AND GENERALIZED CROSS VALIDATION ALGORITHM

In various Computer vision applications for better visualization, interpretation and better recognition, we frequently want to change an image in one style into another one which is a large research area of compressive sensing. Algorithms on compressed sensing are in demand to create better reconstruction of the images while taking less computational time and requiring less storage capacity. In the present work attempts are put in the same direction and brought an algorithm which includes semi coupled dictionary learning (SCDL) model of compressed sensing with Singular Value Decomposition (SVD) based Generalized Cross Validation (GCV) algorithm. Using this algorithm one can find the different value of regularization parameters for different sizes of images and that helps to get better reconstruction of the images with ease while doing experiments on cross style image synthesis problems.

Spatiotemporal Reflectance Fusion via Tensor Sparse Representation

Tradeoffs between the spatial and temporal resolutions of current satellite instruments limit our ability to conduct high-quality and continuous monitoring of the earth’s surface dynamics. Spatiotemporal image fusion has become increasingly necessary to obtain remote sensing images with high spatiotemporal resolution. However, current learning-based methods concentrate on predicting images only from spatial similarity and neglect spectral correlations of remote sensing images, leading to significant spectral information loss. In this article, we develop a novel nonlocal tensor sparse representation-based semicoupled dictionary learning approach (SCDNTSR) for spatiotemporal fusion. In the SCDNTSR method, the spectral correlation and the spatial similarity of the nonlocal similar cubes are simultaneously exploited through the tensor–tensor product-based tensor sparse representation. Furthermore, the semicoupled mapping prior knowledge of sparse coefficients across the high- and low-spatial resolution (HSR\LSR) image spaces is exploited with the coupled dictionary to constrain the similarity of sparse coefficients to improve the prediction performance. In addition, to capture additional prior spatial information, the SCDNTSR provides a new method to determine the degradation relationship between the target HSR and LSR difference images with the help of the known HSR and LSR difference images. The proposed SCDNTSR method was tested on real datasets at both the Coleambally Irrigation Area study site and the Lower Gwydir Catchment study site. Results show that the proposed method outperforms five state-of-the-art methods, especially in maintaining the spectral information, proving the feasibility of integrating the degradation relationship, spatio-spectral-nonlocal correlation, and semicoupled mapping priors of the multisource data into the proposed model.

A Novel Semicoupled Projective Dictionary Pair Learning Method for PolSAR Image Classification

Polarimetric synthetic aperture radar (PolSAR) image classification plays an important role in remote sensing image processing. In recent years, stacked auto-encoder (SAE) has obtained a series of excellent results in PolSAR image classification. The recently proposed projective dictionary pair learning (DPL) model takes both accuracy and time consumption into consideration, and another recently proposed semicoupled dictionary learning (SCDL) model gives a new way to fit different features. Based on the SAE, DPL, and SCDL models, we propose a novel semicoupled projective DPL method with SAE (SAE-SDPL) for PolSAR image classification. Our method can get the classification result efficiently and correctly and meanwhile giving a new method to fit different features. In this paper, three PolSAR images are used to test the performance of SAE-SDPL. Compared with some state-of-the-art methods, our method obtains excellent results in PolSAR image classification.

Investigation of Semi-Coupled Dictionary Learning in 3-D Super Resolution HR-pQCT Imaging

High resolution peripheral quantitative CT (HR-pQCT) permits to investigate bone micro-architecture in vivo . While it is a considerable progress over standard CT for this specific application, the spatial resolution remains so far limited for the investigation of bone topology. In this paper, we investigate super resolution techniques to improve 3-D HR-pQCT images via semi-coupled dictionary learning based on the knowledge of high resolution micro-CT images. We propose a method to select the number of atoms and study the impact of patch sizes. To handle the anisotropy of the 3-D bone structure, we propose a 2.5-D strategy learning low and high resolution dictionaries in a semi-coupled way on the three different directions. The results show that this strategy is superior to the application of the method in 2-D. The bone volume fraction is successfully recovered and the estimation of the connectivity is improved in most test samples. The images have a better quality compared with previously studied methods based on total variation (TV) regularization or a combination of TV and a double-well potential. In the future, we expect to further evaluate the method on a larger data set.

Semi-Coupled Dictionary Learning With Relaxation Label Space Transformation for Video-Based Person Re-Identification

Video-based person reidentification (re-id) is a challenging problem due to much discrepancy between different videos by person pose, illumination, viewpoint change, background clutter, and occlusion within each camera and across different cameras. However, most existing video-based person re-id methods usually focus on dealing with the discrepancy between different cameras and do not fully consider the correlation between different cameras. In this paper, we propose a semicoupled dictionary learning with relaxation label space transformation approach to capture the intrinsic relationship of the same person under different cameras. First, to reduce the discrepancy between different views, we transform the original feature spaces into the common feature space by local Fisher discriminant analysis. Two dictionaries are learned from this common feature space. Second, a relaxation label space is introduced to associate the same person under different views. In this label space, the distance between different persons can be enlarged as much as possible, such that label information has stronger discriminative capability. A single dictionary is learned from the relaxation label space. Finally, in order to further enhance the correlation of the same person between different cameras, we use a pair of transformation matrices which map the coding coefficients learned from the common feature space to the coding coefficients learned from the relaxation label space, respectively. Extensive experimental results on two public iLIDS Video re-IDentification and Person Re-ID 2011 video-based person re-id datasets demonstrate the effectiveness of the proposed method.

SLR: Semi-Coupled Locality Constrained Representation for Very Low Resolution Face Recognition and Super Resolution

Although face recognition algorithms have been greatly successful recently, in real applications of very low-resolution (VLR) images, both super-resolution (SR) and recognition tasks are more challenging than those in high-resolution (HR) images. Given the rare discriminative information in VLR images, the one-to-many mapping relationship between HR and VLR images degrades the SR and recognition performances. In this paper, we propose a novel semi-coupled dictionary learning scheme to promote discriminative and representative abilities for face recognition and SR simultaneously by relaxing coupled dictionary learning. Specifically, we use semi-coupled locality-constrained representation to enhance the consistency between VLR and HR local manifold geometries, thereby overcoming the negative effects of one-to-many mapping. Given the learned task-oriented mapping function, we feed these discriminative features into a collaborative representation-based classifier to output their labels, and combine a locality-induced approach to hallucinate the HR images. Extensive experimental results demonstrate that the proposed approach outperforms a number of state-of-the-art face recognition and SR algorithms.

Spatiotemporal Fusion of Remote Sensing Images with Structural Sparsity and Semi-Coupled Dictionary Learning

Fusion of remote sensing images with different spatial and temporal resolutions is highly needed by diverse earth observation applications. A small number of spatiotemporal fusion methods using sparse representation appear to be more promising than traditional linear mixture methods in reflecting abruptly changing terrestrial content. However, one of the main difficulties is that the results of sparse representation have reduced expressional accuracy; this is due in part to insufficient prior knowledge. For remote sensing images, the cluster and joint structural sparsity of the sparse coefficients could be employed as a priori knowledge. In this paper, a new optimization model is constructed with the semi-coupled dictionary learning and structural sparsity to predict the unknown high-resolution image from known images. Specifically, the intra-block correlation and cluster-structured sparsity are considered for single-channel reconstruction, and the inter-band similarity of joint-structured sparsity is considered for multichannel reconstruction, and both are implemented with block sparse Bayesian learning. The detailed optimization steps are given iteratively. In the experimental procedure, the red, green, and near-infrared bands of Landsat-7 and Moderate Resolution Imaging Spectrometer (MODIS) satellites are put to fusion with root mean square errors to check the prediction accuracy. It can be concluded from the experiment that the proposed methods can produce higher quality than state-of-the-art methods.

An Error-Bound-Regularized Sparse Coding for Spatiotemporal Reflectance Fusion

This paper attempts to demonstrate that addressing the dictionary perturbations and the individual representation of the coupled images can generally result in positive effects with respect to sparse-representation-based spatiotemporal reflectance fusion (SPTM). We propose to adapt the dictionary perturbations with an error-bound-regularized method and formulate the dictionary perturbations to be a sparse elastic net regression problem. Moreover, we also utilize semi-coupled dictionary learning (SCDL) to address the differences between the high-spatial-resolution and low-spatial-resolution images, and we propose the error-bound-regularized SCDL (EBSCDL) model by also imposing an error bound regularization. Two data sets of Landsat Enhanced Thematic Mapper Plus data and Moderate Resolution Imaging Spectroradiometer acquisitions were used to validate the proposed models. The spatial and temporal adaptive reflectance fusion model and the original SPTM were also implemented and compared. The experimental results consistently show the positive effect of the proposed methods for SPTM, with smaller differences in scatter plot distribution and higher peak-signal-to-noise ratio and structural similarity index measures.

Video super-resolution reconstruction based on correlation learning and spatio-temporal nonlocal similarity

A novel video super-resolution reconstruction algorithm based on correlation learning and spatio-temporal nonlocal similarity is proposed in this paper. Objective high-resolution (HR) estimates of low-resolution (LR) video frames can be obtained by learning LR-HR correlation mapping and fusing the spatio-temporal nonlocal similarity information between video frames. First, the LR-HR correlation mapping between LR and HR patches is established based on semi-coupled dictionary learning. With the aim of improving algorithm efficiency while guaranteeing super-resolution quality, LR-HR correlation mapping is performed only for the salient object region, and then an improved visual saliency-based nonlocal fuzzy registration scheme using the pseudo-Zernike moment feature and structural similarity is proposed for spatio-temporal similarity matching and fusion. Visual saliency and self-adaptive regional correlation evaluation strategies are used in spatio-temporal similarity matching to improve algorithm efficiency further. Experimental results demonstrate that the proposed algorithm achieves competitive super-resolution quality compared to other state-of-the-art algorithms in terms of both subjective and objective evaluations.

Learning-Based Video Superresolution Reconstruction Using Spatiotemporal Nonlocal Similarity

Aiming at improving the video visual resolution quality and details clarity, a novel learning-based video superresolution reconstruction algorithm using spatiotemporal nonlocal similarity is proposed in this paper. Objective high-resolution (HR) estimations of low-resolution (LR) video frames can be obtained by learning LR-HR correlation mapping and fusing spatiotemporal nonlocal similarities between video frames. With the objective of improving algorithm efficiency while guaranteeing superresolution quality, a novel visual saliency-based LR-HR correlation mapping strategy between LR and HR patches is proposed based on semicoupled dictionary learning. Moreover, aiming at improving performance and efficiency of spatiotemporal similarity matching and fusion, an improved spatiotemporal nonlocal fuzzy registration scheme is established using the similarity weighting strategy based on pseudo-Zernike moment feature similarity and structural similarity, and the self-adaptive regional correlation evaluation strategy. The proposed spatiotemporal fuzzy registration scheme does not rely on accurate estimation of subpixel motion, and therefore it can be adapted to complex motion patterns and is robust to noise and rotation. Experimental results demonstrate that the proposed algorithm achieves competitive superresolution quality compared to other state-of-the-art algorithms in terms of both subjective and objective evaluations.