Online Dictionary Learning Algorithm Research Articles

Kernel recursive least squares dictionary learning algorithm

An online dictionary learning algorithm for kernel sparse representation is developed in the current paper. In this framework, the input signal nonlinearly mapped into the feature space is sparsely represented based on a virtual dictionary in the same space. At any instant, the dictionary is updated in two steps. In the first step, the input signal samples are sparsely represented in the feature space, using the dictionary that has been updated based on the previous data. In the second step, the dictionary is updated. In this paper, a novel recursive dictionary update algorithm is derived, based on the recursive least squares (RLS) approach. This algorithm gradually updates the dictionary, upon receiving one or a mini-batch of training samples. An efficient implementation of the algorithm is also formulated. Experimental results over four datasets in different fields show the superior performance of the proposed algorithm in comparison with its counterparts. In particular, the classification accuracy obtained by the dictionaries trained using the proposed algorithm gradually approaches that of the dictionaries trained in batch mode. Moreover, in spite of lower computational complexity, the proposed algorithm overdoes all existing online kernel dictionary learning algorithms.

Research on real-time data transmission and multi-scale video image decomposition of embedded optical sensor array based on machine learning

Aiming at the research of real-time data transmission and multi-scale image decomposition of embedded optical sensor array, the principle, method and fusion strategy of multi-sensor image fusion are studied comprehensively, thoroughly and systematically by combining the imaging characteristics of source image with multi-scale geometric analysis tools using machine learning algorithm. A new quality scalable video image coding framework is also proposed in this paper, which is implemented by a multi-scale online dictionary learning algorithm based on structured sparse video signals. For the purpose of different types of images and image fusion, a new high quality scalable video image coding framework based on machine learning algorithm is proposed on the basis of comprehensive analysis of prior information such as imaging mechanism of image sensor and imaging characteristics of source image. A multi-scale online dictionary learning algorithm based on machine learning for sparse video signal structure is proposed. Through the hierarchical structure of wavelet decomposition, the searching domain of online learning is optimized to a hierarchical sparse block, and its sparse representation coefficients are obtained by using machine learning sparse coding idea. The real-time data transmission of embedded optical sensor array based on machine learning and multi-scale image decomposition algorithm proposed in this paper have good fusion performance, which is of great significance for further research and engineering application of image fusion technology.

A New Method for Long-Distance Data Reconstruction in Big Data Environment

In a big data environment, the data from long-term monitoring are of great importance in intelligent diagnoses, prognostic and health management of industrial components. However, in practice, it is difficult to ensure high reconstructive accuracy making the preservation performance of low-frequency information (e.g., periodic fault impulses) stable. To address this issue, this article proposes a new data reconstruction method that combines a smoothing sparse low-rank matrix (SSLRM) with online dictionary learning. This article first presents the design of SSLRM model for decomposing the raw data into a low resonance component (LRC) and a high resonance component (HRC), such that the periodic fault impulse can be isolated and preserved in advance. Then, both LRC and HRC are, respectively, reconstructed using an online dictionary learning algorithm, i.e., Hankel K-singular value decomposition; thus, the reconstructed data can be obtained accordingly. Both simulation case and engineering case are applied to show the effectiveness and practicality of the proposed approach.

Online dictionary learning method for extracting GPR diffractions

AbstractDiffractions in a Ground-Penetrating Radar (GPR) data carry significant responses from near-surface small-scale fractures or karsts. However, this geological information is generally difficult to extract because of the shielding effect of strong reflections from subsurface layers. In order to solve this problem, a GPR diffraction extraction method is proposed for individually separating and imaging of GPR diffractions that incorporates a local plane-wave destruction filter with an online dictionary learning algorithm. The strong reflections are estimated and eliminated by the local plane-wave destruction method and the weak GPR diffractions are extracted by a sparse coding algorithm. In solving this model, a trust-region algorithm is used for accelerating the sparse coding procedures that can scale up gracefully to a large GPR data processing. A numerical experiment demonstrates the good performance of the proposed method in destroying strong reflections and enhancing weak diffractions from small-scale void holes. Real data application further verifies its potential value in resolving fine details of subsurface small-scale buried targets, such as pipes or void holes.

SRAF Insertion via Supervised Dictionary Learning

In modern VLSI design flow, subresolution assist feature (SRAF) insertion is one of the resolution enhancement techniques (RETs) to improve chip manufacturing yield. With aggressive feature size continuously scaling down, layout feature learning becomes extremely critical. In this article, for the first time, we enhance conventional manual feature construction, by proposing a supervised online dictionary learning algorithm for simultaneous feature extraction and dimensionality reduction. By taking advantage of label information, the proposed dictionary learning framework can discriminatively and accurately represent the input data. We further consider SRAF design rules in a global view, and design two integer linear programming models in the post-processing stage of SRAF insertion framework. The experimental results demonstrate that, compared with a state-of-the-art SRAF insertion tool, our framework not only boosts the performance of the machine learning model but also improves the mask optimization quality in terms of edge placement error (EPE) and process variation (PV) band area.

Context-Aware Compressed Sensing of Hyperspectral Image

Traditional hyperspectral imaging technique obtains numerous hyperspectral images (HSIs) with hundreds of spectral bands, leading to high cost in data acquisition, transmission, and storage. Compressed sensing (CS) theory provides a new imaging mechanism, which relies on the assumption that signals can be sparsely represented over a dictionary. By the CS imaging technique, original HSIs can be approximately reconstructed from only a few sampled measurements. In this article, a novel context-aware CS (CACS) method for HSIs is proposed by incorporating contextual prior to the dictionary learning and the sparse reconstruction. First, a patch-based online dictionary learning (ODL) algorithm is developed by introducing a joint sparse constraint. On the one hand, the online dictionary learning mechanism enables a more adaptive representation of HSIs with different scenes than using fixed-basis-based dictionaries, e.g., the discrete cosine transform (DCT) and the discrete wavelet transform dictionaries. On the other hand, the introduced joint sparse constraint promotes the learned dictionary to more sparsely and structurally represent spectral pixels. Then, with the well-learned dictionary, a weighted smoothing regularization is introduced to develop a new sparse reconstruction model. Considering the high spectral–spatial similarity of pixels in a neighborhood, the new sparse reconstruction model will encourage a locally smoothing reconstruction result. In this way, the spectral–spatial structures of the HSI can be well preserved, while possible artifacts can be effectively reduced. Experimental results demonstrate the superiority of the proposed method over some state-of-the-art hyperspectral compressive imaging methods.

Slice-Based Online Convolutional Dictionary Learning

Convolutional dictionary learning (CDL) aims to learn a structured and shift-invariant dictionary to decompose signals into sparse representations. While yielding superior results compared to traditional sparse coding methods on various signal and image processing tasks, most CDL methods have difficulties handling large data, because they have to process all images in the dataset in a single pass. Therefore, recent research has focused on online CDL (OCDL) which updates the dictionary with sequentially incoming signals. In this article, a novel OCDL algorithm is proposed based on a local, slice-based representation of sparse codes. Such representation has been found useful in batch CDL problems, where the convolutional sparse coding and dictionary learning problem could be handled in a local way similar to traditional sparse coding problems, but it has never been explored under online scenarios before. We show, in this article, that the proposed algorithm is a natural extension of the traditional patch-based online dictionary learning algorithm, and the dictionary is updated in a similar memory efficient way too. On the other hand, it can be viewed as an improvement of existing second-order OCDL algorithms. Theoretical analysis shows that our algorithm converges and has lower time complexity than existing counterpart that yields exactly the same output. Extensive experiments are performed on various benchmarking datasets, which show that our algorithm outperforms state-of-the-art batch and OCDL algorithms in terms of reconstruction objectives.

Neutron spectrum unfolding of the multiple activation foils based on sparse representation

An unfolding method based on sparse representation is proposed and applied to unfolding problem of multiple activation foils measurement. The Online Dictionary Learning (ODL) algorithm is applied for sparse representation. The algorithm LARS-EN is used for reconstruction. The Akaike’s Information Criterion (AIC) is applied to select the best coefficient path. Nuclear reactor of Qinshan phase II nuclear power plant (NPP) is considered as the study object and training samples are generated by Monte Carlo code MCNP. By using the unfolding method proposed in this paper, two sets of neutron spectrums are unfolded successfully with acceptable accuracies. The first set contains 4 neutron spectrums of different shapes; and the second set contains 4 neutron spectrums at different positions of the radiation surveillance capsule. Moreover, neutron spectrums of the radiation surveillance capsule can also be unfolded with acceptable accuracies with fewer equations while the two fission detectors 238U and 237Np are removed.

Fusion of United Sparse Principal Component Analysis Dictionary Based on Linear Unmixing Image Technique

Based on the linear unmixing images of different surface objects, online dictionary learning algorithm was utilized to compute the sparse dictionaries for multispectral linear unmixing images and panchromatic images. Principal component analysis (PCA) was then utilized to generate united sparse PCA dictionaries through the extraction of the first principal components of panchromatic images and unmixing image dictionaries. The number of dictionaries is determined to be 480 after taking into consideration of the limitation in computing power and root-mean-square error of restructured images. Based on these dictionaries, orthogonal matching pursuit method was utilized to calculate the sparse coefficients of multispectral and panchromatic images, separately, while nonnegative matrix factorization fusion algorithm was utilized to calculate multispectral and panchromatic sparse coefficients to obtain sparse coefficient of the fusional image on all bands, with the resulted matrix having a size of $480 \times 255\,025$ . These united sparse PCA dictionaries and fusion sparse coefficients were then used to reconstruct the fusional image. Through the analysis of five quantitative indices of fusion assessment, the proposed fusion algorithm has retained the multispectral information of images and enhanced the detailed information in image texture.

Robust Visual Tracking via Smooth Manifold Kernel Sparse Learning

Various sparse-representation-based tracking meth-ods have been proposed to tackle visual tracking problems, and most of them use simple intensity feature as the observation of target object. Moreover, most of them only take into account either global or local image representation and only exploit the underlying relationship among target candidates in a single frame. All of these may make their appearance models less robust to deal with complex scenes. To overcome these problems, we propose a smooth manifold kernel sparse tracker under the framework of particle filter. The proposed method characterizes targets and candidates with region covariance matrix descriptors, and constructs object tracking as a kernel sparse learning model based on symmetric positive-definite (SPD) manifolds. The spatial-temporal interdependencies among candidates and global-local representations of candidates are jointly considered and unified via the kernel sparse learning model. Moreover, in order to make the model more robust, the detection of outlier tasks is also taken into account. To handle the variation of object appearance, we develop a robust and efficient online dictionary learning algorithm on SPD manifolds. Extensive experiments on multiple benchmark datasets demonstrate that our tracker performs favorably against state-of-the-art trackers.

Adaptive online dictionary learning for bearing fault diagnosis.

One of the most common parts to maintain system balance and support the various load in rotating machinery is the rolling element bearing. The breakdown of the element in bearings leads to inefficiency and sometimes catastrophic events across various industries. The main challenge over the last few years for fault diagnosis of bearings is the early detection of fault signature. In this paper, an adaptive online dictionary learning algorithm is developed for early fault detection of bearing elements. The dictionary is trained using a set of vibration signal from a heavily damaged bearing. The enveloped signal of the bearing is obtained using the Kurtogram and split into several sections. The K-SVD algorithm is implemented to the dictionaries corresponding to the enveloped signal. OMP is implemented with the calculated dictionaries to obtain the sparse representation of the testing signal. Then the envelope analysis is implemented to obtain the fault signal from the recovered signal by the dictionaries. The adaptive algorithm is added to the dictionary learning to update the dictionary based on newly acquired data with the weighted least square method. Without retraining the dictionaries using the K-SVD algorithm, the computation speed is significantly improved. The proposed algorithm is compared with a traditional dictionary learning algorithm to show the improvement in detection of new fault frequency and improved signal to noise ratio.

Identifying crop planting areas using Fourier-transformed feature of time series MODIS leaf area index and sparse-representation-based classification in the North China Plain

ABSTRACTAccurate information of crop distribution is vital for agricultural industry and food security, which gives rise to a strong demand for timely crop mapping. This study applies a method that combines sparse representation and harmonic characteristics derived from time series Moderate Resolution Imaging Spectroradiometer (MODIS) leaf area index (LAI) to identify crop planting areas to the north of the Yellow River in the North China Plain, where winter wheat and maize are widely cultivated. The upper envelope Savitzky–Golay filter was applied on the yearly time series MODIS LAI pixel by pixel to minimize the effects of anomalous values caused by atmospheric variability and cloud contamination. The Fourier transform method was then employed to extract the key parameters from the Savitzky–Golay filtered LAI. Totally about 11 parameters were extracted, including the amplitudes of the terms 0–5 and the phases of the terms 1–5 of the Savitzky–Golay filtered LAI, and were taken as the features for the crop identification. Based on the training samples of the identified crops, which were obtained through field campaigns and Google Earth images, the online dictionary learning algorithm was applied to construct the dictionary for identifying the crops. With the dictionary, orthogonal matching pursuit algorithm was applied to obtain sparse representation coefficients of the identified samples. Then crops were identified using the minimum reconstruction errors which were calculated by the dictionary and the sparse representation coefficients of each category. Winter wheat, spring maize, summer maize, cotton, and orchard planting areas from 2003 to 2016 in the study area were identified. The accuracy of the identification was evaluated by the confusion matrix. Average overall identification accuracy in the 14 years was 78.67%, with a kappa coefficient (κ) of 0.75. Annual overall accuracy from 2003 to 2016 was from 70.57% to 83.71%, and κ were from 0.66 to 0.81. These results indicate that the accuracy of crop identification is high. Compared with traditional classification methods, such as supervised classification, the approach developed in this study is feasible and efficient for identifying crop distribution in the study area. Repeated and expensive ground sampling year after year is not needed, which effectively reduces computation cost. The findings of this study can provide valuable implications for crop condition monitoring and yield estimation in the region.

Robust object tracking via constrained online dictionary learning

Robust object tracking has widespread applications in human motion analysis systems, but it is challenging due to various factors, such as occlusion, illumination variation, and complex backgrounds. In this paper, we present a novel tracking method on the basis of a constrained online dictionary learning algorithm. Some existing tracking methods cannot consider background effects and thus have weak discriminative ability. Moreover, some dictionary learning-based tracking methods directly collect target templates and background templates as positive and negative dictionaries, respectively. The main issue is that the dictionaries cannot effectively represent the target and background and handle appearance changes. Thus, a constrained online dictionary learning algorithm is proposed to obtain a discriminative dictionary, which can ensure that the proposed tracker has good discriminative ability in distinguishing targets from complex backgrounds. Experimental results show that the proposed algorithm performs favorably against other state-of-the-art methods in terms of accuracy and robustness.

DUK-SVD: dynamic dictionary updating for sparse representation of a long-time remote sensing image sequence

Sparse representations of data or signals have drawn considerable attentions in the past decade. In this paper, we focus on the problem of training high-efficacy dictionaries for remote sensing images of massive long-time sequences. By extending the classical K-SVD, we propose a new dictionaries learning algorithm. Different from K-SVD, in the proposed incremental K-SVD algorithm, we selectively train a certain number of atoms when each new batch of sample data are added into the training process; current dictionary are replenished by the selected and enhanced atoms. The new atoms are initialized by information entropy. Meanwhile, we introduce an uncertainty metric to determine whether or not new atoms should be added into the current dictionary. To efficiently and sparsely represent the long-time sequence data set, we also de-correlate the dictionary based on new atoms by introducing a mutual coherence constraint into the atom updating stage. The method presented in this paper aims to adaptively and dynamically train the dictionary from big data. Two other state-of-the-art dictionary learning methods such as online dictionary learning (ODL) and recursive least squares dictionary learning algorithm (RLS-DLA) who also could train the dictionary using relatively large data, are comprehensively compared with the proposed algorithm in both sparse model and error model. In the sparse model, the reconstruction error of the DUK-SVD dictionary was smaller than ODL and RLS-DLA. In the error model, the sparsity of the DUK-SVD was higher than ODL and RLS-DLA. We can also observe that in the sparse model the proposed DUK-SVD often consume fewer computing time than ODL.

Medical image fusion based on sparse representation of classified image patches

Abstract Medical image fusion is one of the hot research in the field of medical imaging and radiation medicine, and is widely recognized by medical and engineering fields. In this paper, a new fusion scheme for medical images based on sparse representation of classified image patches is proposed. In this method, first, the registered source images are divided into classified patches according to the patch geometrical direction, from which the corresponding sub-dictionary is trained via the online dictionary learning (ODL) algorithm, and the least angle regression (LARS) algorithm is used to sparsely code each patch; second, the sparse coefficients are combined with the “choose-max” fusion rule; Finally, the fused image is reconstructed from the combined sparse coefficients and the corresponding sub-dictionary. The experimental results showed that the proposed method outperforms other methods in terms of both visual perception and objective evaluation metrics.

Anomaly detection using sparse reconstruction in crowded scenes

In this paper, we propose an algorithm of anomaly detection in crowded scenes by using sparse representation over the normal bases. First, the histogram of maximal optical flow projection (HMOFP) features are extracted from a set of normal training data. Then, the online dictionary learning algorithm is used to train an optimal dictionary with proper redundancy, which is better than the dictionary simply composed by the HMOFP features of the whole training data. In order to detect the normalness of a frame, the l1-norm of the sparse reconstruction coefficients is used as the Reconstruction Coefficient Sparsity (RCS). Our algorithm is effective for both global abnormal events (GAE) and local abnormal events (LAE). We evaluate our method on three benchmark datasets-the UMN dataset, the PETS2009 dataset and the UCSD Ped1 dataset. Compared with the most popular methods, experimental results show that our algorithm achieves good results especially for the pixel-level local abnormal event localization.

Sparse Representation With Spatio-Temporal Online Dictionary Learning for Promising Video Coding.

Classical dictionary learning methods for video coding suffer from high computational complexity and interfered coding efficiency by disregarding its underlying distribution. This paper proposes a spatio-temporal online dictionary learning (STOL) algorithm to speed up the convergence rate of dictionary learning with a guarantee of approximation error. The proposed algorithm incorporates stochastic gradient descents to form a dictionary of pairs of 3D low-frequency and high-frequency spatio-temporal volumes. In each iteration of the learning process, it randomly selects one sample volume and updates the atoms of dictionary by minimizing the expected cost, rather than optimizes empirical cost over the complete training data, such as batch learning methods, e.g., K-SVD. Since the selected volumes are supposed to be independent identically distributed samples from the underlying distribution, decomposition coefficients attained from the trained dictionary are desirable for sparse representation. Theoretically, it is proved that the proposed STOL could achieve better approximation for sparse representation than K-SVD and maintain both structured sparsity and hierarchical sparsity. It is shown to outperform batch gradient descent methods (K-SVD) in the sense of convergence speed and computational complexity, and its upper bound for prediction error is asymptotically equal to the training error. With lower computational complexity, extensive experiments validate that the STOL-based coding scheme achieves performance improvements than H.264/AVC or High Efficiency Video Coding as well as existing super-resolution-based methods in rate-distortion performance and visual quality.

Correlation Based Online Dictionary Learning Algorithm

The goal of dictionary learning algorithms is to learn a set of atoms called dictionary from a set of training data such that each training data can be represented sparsely by the dictionary. Most dictionary learning algorithms use two alternating steps, sparse coding and dictionary update, to solve this problem. In this paper, we propose a new online dictionary learning algorithm with a novel dictionary update step. In the new update method, only the atoms involved in the sparse representation of new training data are adaptively updated. The adaptive update also includes using the previous training data that have common atoms with the new training data in their sparse representation. The proposed algorithm reduces the computational complexity by reducing the number of atoms updating at each iteration and the number of training data contributing in the dictionary update. In the proposed algorithm, different weights are given to training data, so each has the proper level of influence in the dictionary update. The simulation results show this improves the performance of the algorithm, in convergence speed and representation error. We also show that our algorithm has the ability of full recovery of dictionary with only one trial over training data and does not need any dictionary pruning to remove the unused atoms of the dictionary. We compare our proposed algorithm with well-known online and batch based methods using synthetic and autoregressive data.

Trainlets: Dictionary Learning in High Dimensions

Sparse representation has shown to be a very powerful model for real world signals, and has enabled the development of applications with notable performance. Combined with the ability to learn a dictionary from signal examples, sparsity-inspired algorithms are often achieving state-of-the-art results in a wide variety of tasks. These methods have traditionally been restricted to small dimensions mainly due to the computational constraints that the dictionary learning problem entails. In the context of image processing, this implies handling small image patches. In this work we show how to efficiently handle bigger dimensions and go beyond the small patches in sparsity-based signal and image processing methods. We build our approach based on a new cropped Wavelet decomposition, which enables a multi-scale analysis with virtually no border effects. We then employ this as the base dictionary within a double sparsity model to enable the training of adaptive dictionaries. To cope with the increase of training data, while at the same time improving the training performance, we present an Online Sparse Dictionary Learning (OSDL) algorithm to train this model effectively, enabling it to handle millions of examples. This work shows that dictionary learning can be up-scaled to tackle a new level of signal dimensions, obtaining large adaptable atoms that we call Trainlets.

Machinery vibration signal denoising based on learned dictionary and sparse representation

Mechanical vibration signal denoising has been an import problem for machine damage assessment and health monitoring. Wavelet transfer and sparse reconstruction are the powerful and practical methods. However, those methods are based on the fixed basis functions or atoms. In this paper, a novel method is presented. The atoms used to represent signals are learned from the raw signal. And in order to satisfy the requirements of real-time signal processing, an online dictionary learning algorithm is adopted. Orthogonal matching pursuit is applied to extract the most pursuit column in the dictionary. At last, denoised signal is calculated with the sparse vector and learned dictionary. A simulation signal and real bearing fault signal are utilized to evaluate the improved performance of the proposed method through the comparison with kinds of denoising algorithms. Then Its computing efficiency is demonstrated by an illustrative runtime example. The results show that the proposed method outperforms current algorithms with efficiency calculation.