Dictionary Learning Research Articles

A Projected OMP-Hybridized Discriminative K-SVD-Based Dictionary Learning Algorithm for Human Activity Recognition From Accelerometer Signals

A Projected OMP-Hybridized Discriminative K-SVD-Based Dictionary Learning Algorithm for Human Activity Recognition From Accelerometer Signals

Analysis for sparse channel representation based on dictionary learning in massive MIMO systems

AbstractThe accuracy analysis of dictionary sparse representation for channels in massive MIMO systems is a relatively unexplored field. Existing research has primarily focused on investigating the accuracy of dictionary sparse representation using simulation in massive MIMO systems, but has not provided quantitative accuracy analysis. To address this gap, the correlation numerical proportional factor is proposed to represent the accuracy performance of non‐zero elements in the coefficient matrix. Additionally, a qualitative analytical formula for dictionary sparse representation accuracy is provided and an optimal upper bound for the correlation numerical proportional factor is established. Furthermore, the innovation indicates that the accuracy of dictionary sparse representation is mainly influenced by the cross‐correlation between the pilots matrix and the dictionary matrix, as well as sparsity. The author has also developed a method for minimizing the correlation numerical proportional factor. In order to obtain an optimal sparse representation coefficient matrix, a cross‐correlation matrix is constructed and an analytical expression is derived for it as well as its use as an optimal hard decision threshold is determined. Finally, a sparse representation coefficient optimization algorithm is proposed using this optimal threshold. Simulation results demonstrate that this algorithm can significantly improve channel sparse dictionary representation accuracy.

Alternative Learning Paradigms for Image Quality Transfer

Image Quality Transfer (IQT) aims to enhance the contrast and resolution of low-quality medical images, e.g. obtained from low-power devices, with rich information learned from higher quality images. In contrast to existing IQT methods in the literature which adopt supervised learning frameworks, in this work, we propose two novel formulations of the IQT problem. The first approach uses an unsupervised learning framework, whereas the second is a combination of both supervised and unsupervised learning. The unsupervised learning approach considers a sparse representation (SRep) and dictionary learning model, which we call IQT-SRep, whereas the combination of supervised and unsupervised learning ap- proach is based on deep dictionary learning (DDL), which we call IQT-DDL. The IQT-SRep approach trains two dictionaries using a sparse representation model using pairs of low- and high-quality volumes. Subsequently, the sparse representation of a low-quality block, in terms of the low-quality dictionary, can be directly used to recover the corresponding high-quality block using the high-quality dictionary. On the other hand, the IQT-DDL ap- proach explicitly learns a high-resolution dictionary to upscale the input volume, while the entire network, including high dictionary generator, is simultaneously optimised to take full advantage of deep learning methods. The two models are evaluated using a low-field mag- netic resonance imaging (MRI) application aiming to recover high-quality images akin to those obtained from high-field scanners. Experiments comparing the proposed approaches against state-of-the-art supervised deep learning IQT method (IQT-DL) identify that the two novel formulations of the IQT problem can avoid bias associated with supervised meth- ods when tested using out-of-distribution data that differs from the distribution of the data the model was trained on. This highlights the potential benefit of these novel paradigms for IQT.

SimCDL: A Simple Framework for Contrastive Dictionary Learning

In this paper, we propose a novel approach to the dictionary learning (DL) initialization problem, leveraging the SimCLR framework from deep learning in a self-supervised manner. Dictionary learning seeks to represent signals as sparse combinations of dictionary atoms, but effective initialization remains challenging. By applying contrastive learning, we encourage similar representations for augmented versions of the same sample while distinguishing between different samples. This results in a more diverse and incoherent set of atoms, which enhances the performance of DL applications in classification and anomaly detection tasks. Our experiments across several benchmark datasets demonstrate the effectiveness of our method for improving dictionary learning initialization and its subsequent impact on performance in various applications.

Deep convolutional sparse dictionary learning for bearing fault diagnosis under variable speed condition

Dictionary learning is a usual method in the field of machinery fault diagnosis, but it requires that the rotating speed conditions of training set and test set are the same and constant. When the speed condition of test set is different from that of training set or one of them is time-vary, normal dictionary learning is difficult to get a precise sparse representation. A special dictionary model named convolutional sparse dictionary (CSD) can overcome the influence from variable speed conditions by atoms locally shifting in the sample's dimension, which is beneficial to capture the local fault features in the signal no matter how the speed changes. However, there are both large features and small features in the mechanical vibration signal, and several continuous small features can also form a large feature. The problem is that CSD can only locally optimize the signal at a fixed scale, so the features of other scales cannot be optimized. To solve this problem, this paper proposes a model named deep convolutional sparse dictionary (DCSD) to extract bearing fault features under variable speed conditions, which is improved from CSD. DCSD has multiple dictionary layers, where each layer is a CSD, but the atom's dimensions are different in each layer. The larger the number of layer is, the larger the atom's dimension is, and the sparse representation result of each layer is used to train the next dictionary layer. Through simulations cases and experimental cases under variable speed conditions, it is proved that DCSD has better performances than CSD in the fault diagnosis.

Unsupervised 3D seismic data reconstruction using a weighted-attentive deep-learning framework

The physical and cost limitations of seismic data acquisition often result in the spatial undersampling of data, which has a detrimental impact on subsequent data processing. Seismic data reconstruction plays a crucial role in recovering missing traces arising from the acquisition process. In recent years, deep learning (DL) has emerged as an intelligent solution for this purpose. We introduce an innovative approach called the weighted-attentive DL framework for unsupervised 3D seismic data reconstruction, which combines an attentive transformer network (ATNet) with a conventional projection onto convex sets (POCS) algorithm. Our framework follows the plug-and-play concept, requiring only the original subsampled data to achieve robust performance. It accomplishes this by iteratively updating ATNet parameters, wherein ATNet serves as the reconstruction operator during the DL process. This approach allows us to simultaneously recover missing traces and filter out random noise, with a key feature being its enhanced attention to the observed signal compared with convolutional neural networks. In addition, we incorporate the POCS algorithm into our framework to introduce a linear decay weight for the aliasing space containing noise and signal during the reconstruction process. We rigorously evaluate our method against four existing approaches: adaptive POCS, fast dictionary learning sequential generalized K-means, optimally damped rank-reduction, and DenseNet methods. Our comparative experiments, conducted on synthetic and field data examples, clearly demonstrate the substantial improvements achieved by our method in terms of reconstruction and denoising when compared with the four benchmarked methods.

A Novel Dictionary Learning Algorithm Based on Prior Knowledge for fMRI Data Analysis

ABSTRACTTask‐based functional magnetic resonance imaging (fMRI) has been widely utilized for brain activation detection and functional network analysis. In recent years, the K‐singular value decomposition (K‐SVD) algorithm has gained increasing attention in the research of fMRI data analysis methods. In this study, we propose a novel temporal feature region‐growing constrained K‐SVD algorithm that incorporates task‐based fMRI temporal prior knowledge and utilizes a region‐growing algorithm to infer potential activation locations. The algorithm incorporates temporal and spatial constraints to enhance the detection of brain activation. Specifically, this paper improves the three stages of the traditional K‐SVD algorithm. First, in the dictionary initialization stage, the automatic target generation process with an independent component analysis algorithm is utilized in conjunction with prior knowledge to enhance the accuracy of initialization. Second, in the sparse coding stage, the region‐growing algorithm is employed to infer potential activation locations based on temporal prior knowledge, thereby imposing spatial constraints to limit the extent of activation regions. Finally, in the dictionary learning stage, soft constraints and low correlation constraints are applied to reinforce the consistency with prior knowledge and enhance the robustness of learning for task‐related atoms. The proposed method was validated on simulated and real fMRI data, showing superior performance in detecting brain activation compared with traditional methods. The results indicate that the algorithm accurately identifies activated brain regions, providing an effective approach for studying brain function in clinical applications.

A fingerprint dictionary processing approach in indoor localization system based on wi-fi

Indoor localization using Wi-Fi fingerprinting based on Received Signal Strength (RSS) has gained widespread attention due to its immunity to external factors and ability to penetrate obstacles. The localization process involves an offline phase for building a radio map and an online phase for matching location queries. Existing matching algorithms often prioritize enhancing online phase accuracy, overlooking the importance of offline data preprocessing, which can negatively impact overall performance. This study introduces a novel approach called Fingerprint Dictionary Preprocessing (FDP) that employs Convolutional Dictionary Learning (CDL) to process radio map data. CDL learns a set of kernels capturing site characteristics, representing RSS values from Access Points (APs) in a sparse manner. The proposed FDP system compresses data through feature learning, reducing storage requirements for data transmission. In the online phase, CDL is utilized for assisting matching fingerprints against the learned dictionary, accurately locating users. The contributions of the FDP system presenting a cost-effective and practical solution for indoor localization, addressing the challenges associated with large data collection and multi-dimensional data requirements, making it a promising approach for real-world applications. We conducted experiments in two real indoor environments, and the results indicated that the proposed FDP system, whether applied to the original radio map or the preprocessed fingerprint database, led to improved localization accuracy and reduced localization time.

Massive MIMO uplink and downlink joint representation based on couple dictionary learning

AbstractThe challenge of jointly representing both the uplink (UL) and downlink (DL) in massive multiple input multiple output (MIMO) systems have been tackled. Considering the angular reciprocity, a couple dictionary learning (CDL) support to enhance performance and address high complexity has been introduced. This approach minimizes the number of pilots and improves accuracy. Currently, accuracy analysis of UL/DL representation primarily relies on simulation. To bridge this gap, a proportion factor (PF) operator is proposed for CDL to assess accuracy. Specifically, a qualitative analysis formula is provided for accuracy and an optimal upper bound is established. Through theoretical proof, it is demonstrated that the accuracy of CDL for representation is mainly influenced by the cross‐correlation between the pilot matrix and the dictionary matrix. Inspired by PF operator, an optimal couple dictionary learning (OCDL) algorithm using singular value decomposition (SVD) is introduced to obtain dictionary updating, aiming at high‐precision representation. By establishing normalized mean squared error (NMSE), successful representation ratio, bit error rate (BER), and constellation performance, an OCDL algorithm that outperforms existing methods is showcased and channel representation accuracy is enhanced significantly.

Dictionary Learning Method Based on K-Sparse Approximation and Orthogonal Procrustes Analysis for Reconstruction in Bioluminescence Tomography.

Bioluminescence tomography (BLT) is one kind of noninvasive optical molecular imaging technology, widely used to study molecular activities and disease progression inside live animals. By combining the optical propagation model and inversion algorithm, BLT enables three-dimensional imaging and quantitative analysis of light sources within organisms. However, challenges like light scattering and absorption in tissues, and the complexity of biological structures, significantly impact the accuracy of BLT reconstructions. Here, we propose a dictionary learning method based on K-sparse approximation and Orthogonal Procrustes analysis (KSAOPA). KSAOPA uses an iterative alternating optimization strategy, enhancing solution sparsity with k-coefficients Lipschitzian mappings for sparsity(K-LIMAPS) in the sparse coding stage, and reducing errors with Orthogonal Procrustes analysis in the dictionary update stage, leading to stable and precise reconstructions. We assessed the method performance through simulations and in vivo experiments, which showed that KSAOPA excels in localization accuracy, morphological recovery, and in vivo applicability compared to other methods.

Environmental sound classification using raw-audio based ensemble framework

Environmental sound classification (ESC) aims to automatically recognize audio recordings from the underlying environment, such as "urban park" or "city centre". Most of the existing methods for ESC use hand-crafted time-frequency features such as log-mel spectrogram to represent audio recordings. However, the hand-crafted features rely on transformations that are defined beforehand and do not consider the variability in the environment due to differences in recording conditions or recording devices. To overcome this, we present an alternative representation framework by leveraging a pre-trained convolutional neural network, SoundNet, trained on a large-scale audio dataset to represent raw audio recordings. We observe that the representations obtained from the intermediate layers of SoundNet lie in low-dimensional subspace. However, the dimensionality of the low-dimensional subspace is not known. To address this, an automatic compact dictionary learning framework is utilized that gives the dimensionality of the underlying subspace. The low-dimensional embeddings are then aggregated in a late-fusion manner in the ensemble framework to incorporate hierarchical information learned at various intermediate layers of SoundNet. We perform experimental evaluation on publicly available DCASE 2017 and 2018 ASC datasets. The proposed ensemble framework improves performance between 1 and 4 percentage points compared to that of existing time-frequency representations.

Double-dictionary learning unsupervised feature selection cooperating with low-rank and sparsity

The feature selection algorithm based on dictionary learning has been widely studied for its excellent interpretability. In the feature selection process, many algorithms only consider the global or local geometric structure information of the original data. A few algorithms that utilize the global and local information together do not actually use the two parts synchronously. Because of this, the information of the two parts cannot be fully utilized reasonably. For this reason, a novel feature selection algorithm, double-dictionary learning unsupervised feature selection cooperating with low-rank and sparsity (LRSDFS), is proposed in this paper. First, LRSDFS improves the traditional dictionary learning by synchronously reconstructing the original dataset into two dictionaries simultaneously. Second, the low-rank and sparsity constraint are applied to the two dictionaries, so that the reconstructed dictionary can retain the global and local information of the original data simultaneously. Finally, the global and local information are weighted to realize the feature selection of the dataset, making the selected features more reasonable and interpretable. LRSDFS is compared with seven state of the art algorithms, including baseline, and evaluated on nine publicly available benchmark datasets. The results show that LRSDFS is more efficient than other unsupervised feature selection algorithms.

Wavelet-based spatiotemporal sparse quaternion dictionary learning for reconstruction of multi-channel vibration data

Accurate reconstruction of multi-source data information plays an essential role in remote intelligent operation, and prognostic and health maintenance (RIO-PHM) of industrial systems. However, traditional signal reconstruction methods such as compressed sensing fail to capture the spatio-temporal characteristics and coupling nature of the multi-source or multi-channel data. To alleviate this bottleneck, in this paper, a new wavelet-based spatiotemporal sparse quaternion dictionary learning (WSTS-QDL) algorithm is formulated for the reconstruction of multi-channel vibration data for the first time. Specifically, the wavelet-based spatiotemporal sparse low-rank matrix (SLRM) algorithm is elaborated and the sparse coefficient matrix of the multi-channel signals can be estimated using the split Bregman iteration (SBI) technique. Then, quaternion-based dictionary learning is introduced for multi-channel data reconstruction according to the updated sparse coefficient matrix during quaternion dictionary learning. Eventually, two equipmental scenarios including run-to-failure data of rolling bearing in bearing test bench and vibration signal of the failured bearing in corn thresher, are utilized for verification. The experimental results demonstrate that the highest recovery accuracy performance with scoring function and the lowest errors are achieved by the proposed method in contrast with the state-of-the-art benchmarks such as orthogonal matching pursuit (OMP) method and spatiotemporal sparse Bayesian learning (SSBL), and the waveform of phase space trajectory and points cloud distribution of the Poincare section in the original signal are similar/same to that obtained by the proposed method.

De-noising magnetotelluric data based on machine learning

The magnetotelluric (MT) sounding is a common geophysical exploration technique, but it is highly polluted by various types of cultural noise. In the realm of MT data processing, traditional techniques often rely on the quality of the measured MT data. Conventional MT time domain denoising methods tend to eliminate valuable signals, potentially leading to unreliable resistivity estimates. To address this concern, we propose employing machine learning to effectively suppress strong noise interference in MT data, thereby preventing the loss of valuable signals. We augment this approach with mathematical morphological filtering (MMF) to capture low-frequency signals, preserving their integrity. We constructed a signal sample library based on a substantial volume of signal samples. Through consistent training, we establish a support vector machine (SVM) classification model that distinguishes high-quality signal fragments from noisy signals. Subsequently, we use adaptive K-singular value decomposition (K-SVD) dictionary learning to extract noise profiles and suppress noisy signals. To validate the feasibility of our method, we apply machine learning to measured data from two distinct observation areas. The measured data were analyzed and processed, and the results were compared with the robust results. This method can effectively eliminate large-scale strong interference in time domain sequences and preserve more low-frequency slow change information and high-quality signals in the reconstructed signals. The apparent resistivity phase curve of synthetic data is smoother and more continuous, and the data quality in the low-frequency range is significantly improved. The results can more accurately and reliably reflect underground electrical structure information.

A dictionary learning based unsupervised neural network for single image compressed sensing

In the field of Compressed Sensing (CS), the sparse representation of signals and the advancement of reconstruction algorithms are two critical challenges. However, conventional CS algorithms often fail to sufficiently exploit the structured sparsity present in images and suffer from poor reconstruction quality. Most deep learning-based CS methods are typically trained on large-scale datasets. Obtaining a sufficient number of training sets is challenging in many practical applications and there may be no training sets available at all in some cases. In this paper, a novel deep Dictionary Learning (DL) based unsupervised neural network for single image CS (dubbed DL-CSNet) is proposed. It is an effective trainless neural network that consists of three components and their corresponding loss functions: 1) a DL layer that consists of multi-layer perceptron (MLP) and convolution neural networks (CNN) for latent sparse features extraction with the L1-norm sparsity loss function; 2) an image smoothing layer with the Total Variation (TV) like image smoothing loss function; and 3) a CS acquisition layer for image compression, with the Mean Square Error (MSE) loss function between the original image compression and the reconstructed image compression. In particular, the proposed DL-CSNet is a lightweight and fast model that does not require datasets for training and exhibits a fast convergence speed, making it suitable for deployment in resource-constrained environments. Experiments have demonstrated that the proposed DL-CSNet achieves superior performance compared to traditional CS methods and other unsupervised state-of-the-art deep learning-based CS methods.

A TMSBL underwater acoustic channel estimation method based on dictionary learning denoising

The shallow sea underwater acoustic channel exhibits a significant sparse multipath structure. The temporally multiple sparse Bayesian learning (TMSBL) algorithm can effectively estimate this sparse multipath channel. However, the complexity of the algorithm is high, the signal-to-noise ratio (SNR) of shallow-sea underwater acoustic communication is low, and the estimation performance of the TMSBL algorithm is greatly affected by noise. To address this problem, an improved TMSBL underwater acoustic channel estimation method based on a dictionary learning noise reduction algorithm is proposed. Firstly, the K-Singular Value Decomposition (K-SVD) dictionary learning method is used to reduce the noise of the received pilot matrix, reducing the influence of noise on the signal. Then, the Generalized Orthogonal Matching Pursuit (GOMP) channel estimation method is combined to obtain a priori information such as the perceptual matrix and hyperparameter matrix for TMSBL channel estimation; and the noise variance is obtained by using the null subcarrier calculation instead of iteratively updating the noise variance in the TMSBL, to improve the estimation accuracy and reduce the algorithmic complexity. Finally, the TMSBL channel estimation method is used to estimate the underwater acoustic channels of different symbols jointly. The simulation results show that the normalized mean square error of the channel estimation of the improved TMSBL method is reduced by about 92.2% compared with the TMSBL algorithm, obtaining higher estimation accuracy; running time is reduced by about 45.6%, and there is also better performance in terms of the running speed, which provides a reference for underwater acoustic channel estimation.

Evaluating the Typographical Features of "Word of the Day" in Online English Learner's Dictionaries Based on a Multimodal Communicative Approach

The emergence of online electronic dictionaries has brought convenience to English learners. The "Big Five" in English learning dictionaries have kept up with the pace of the times, and have established their own websites which are now working as online dictionaries, so that users can use any device which is connected to the Internet to access consultations for words. On the basis of previous studies, this paper aims to focus the research on "Word of the Day", a section unique to nowadays online dictionaries, and conducts an analysis of this section in the "Big Five" learner's dictionaries from the perspective of multimodality and typographical features.

Pengembangan Kamus Mini untuk Membantu Pembelajaran Bahasa Arab Kelas VIII di SMP Datok Sulaiman Palopo

This research aims to determine the development process, level of validity and level of practicality of small Arabic dictionary media which can be used to support the Arabic language learning process, especially in vocabulary learning. The development of this small Arabic dictionary media is adapted to the learning curriculum and teaching materials for class VIII Junior High Schools. This type of research is research and development (R&D) by adapting the 4D development model which consists of four stages, namely definition, design, development and dissemination. The subjects used in this research were 18 class VIII students at Datok Sulaiman Junior High School. Data collection techniques include observation, interviews, validation sheets, practicality test sheets and student response questionnaires. The result of this development research is a small Arabic dictionary for class VIII students. The product validation test results show that: 1) the material expert validation assessment obtained a score of 96.36%; 2) media expert validation assessment obtained a score of 86.66%; 3) and the practicality test carried out by learning experts obtained a score of 80%. Based on the results of product trials by three experts, it shows that the small Arabic dictionary learning media is very valid and practical to be applied in Arabic language learning.

Hyperspectral anomaly detection based on weighted low-rank sparse dictionary learning

The geometric model-based hyperspectral anomaly detection techniques have garnered a lot of interest recently. These methods are predicated on the assumption that the background can be represented by a dictionary of spectral vectors, but the anomaly cannot. Building an objective model with high representational capabilities and obtaining precise background modeling are therefore key aspects of this type of method. The background dictionary is also readily tainted by anomalies due to the limitations of the current approaches, making the detection findings less reliable. To address the above problems, this paper proposes a weighted low-rank sparse dictionary learning method (WLSDL). This model organically combines sparse representation with low-rank representation in order to effectively depict the intricate background distribution. The weights relating to the eigenvalues of the image are designed by mining the physical characteristics of the HSI. This can enhance the nuclear norm’s capacity for representation, resulting in a background dictionary with more representativeness. Furthermore, the capped norm constraint is incorporated in the objective function to decrease the effect of anomalies and noises on background modeling, hence minimizing anomaly pollution on the background dictionary. The residual image effectively increases the accuracy of anomaly identification since it makes it easier to distinguish between anomalies and background. We use three hyperspectral datasets to evaluate the proposed method. The experimental findings demonstrate the effectiveness and superiority of the proposed method over state-of-the-art methods.

Skin cancer classification using non-local means denoising and sparse dictionary learning based CNN

Skin conditions are becoming increasingly prevalent across the world in current times. With the rise in dermatological disorders, there is a need for computerized techniques that are completely noninvasive to patients’ skin. As a result, deep learning models have become standard for the computerized detection of skin diseases. The performance efficiency of these models improves with access to more data with their primary aim being image classification. In this paper, we present a skin disease detection methodology using image processing techniques, non-local means denoising and convolutional neural network (CNN) backed by sparse dictionary learning. Here, the major benefit of using NLM denoising followed by sparse dictionary learning with CNNs in image classification lies in leveraging a multi-stage approach that enhances the quality of input data, extracts meaningful and discriminative features, and improves the overall performance of the classification model. This combined approach addresses challenges such as noise robustness, feature extraction, and classification accuracy, making it particularly effective in complex image analysis tasks. For denoising, the average Peak Signal to Noise Ratio (PSNR) obtained for images from HAM-10000 dataset is 33.59 dB. For the ISIC-2019 dataset, the average PSNR for the train folder is 34.37 dB, and for the test folder it is 34.39 dB. The deep learning network is trained for the analysis of skin cancer images using a CNN model and is achieving acceptable results in classifying skin cancer types. The datasets used contain high-resolution images. After all the tests, the accuracy obtained is 85.61% for the HAM-10000 dataset and 81.23% for the ISIC-2019 dataset, which is on par with existing approaches validated by benchmarking results.