Sparse Vector Research Articles

Sparse Bayesian inference with regularized Gaussian distributions *

Regularization is a common tool in variational inverse problems to impose assumptions on the parameters of the problem. One such assumption is sparsity, which is commonly promoted using lasso and total variation-like regularization. Although the solutions to many such regularized inverse problems can be considered as points of maximum probability of well-chosen posterior distributions, samples from these distributions are generally not sparse. In this paper, we present a framework for implicitly defining a probability distribution that combines the effects of sparsity imposing regularization with Gaussian distributions. Unlike continuous distributions, these implicit distributions can assign positive probability to sparse vectors. We study these regularized distributions for various regularization functions including total variation regularization and piecewise linear convex functions. We apply the developed theory to uncertainty quantification for Bayesian linear inverse problems and derive a Gibbs sampler for a Bayesian hierarchical model. To illustrate the difference between our sparsity-inducing framework and continuous distributions, we apply our framework to small-scale deblurring and computed tomography examples.

Multi-target Localization Based on an Improved FOCUSS Algorithm with Missing Information

The situation where the observation stations or the sensors cannot fully detect the targets is called missing information, which makes it difficult to locate targets. Due to its sparsity, Compressed Sensing (CS) can be applied to multi-target localization with missing information. An improved Focal Undetermined System Solver (IFOCUSS) algorithm is proposed, by dynamically updating the regularized factor that has a significant impact on the convergence of iteration processes. What’s more, a re-weighting method is adopted for the weighting matrix to improve convergence property and accelerate convergence, and a new method of choosing correct items from a sparse estimation vector is proposed to improve the positioning accuracy. Several simulations corroborate its notable reconstruction performance, which is illustrated by correctly estimating five targets with only 11 sensors on a moving station, achieving better convergence at lower signal-to-noise ratios (such as 12 dB) with about half the run time of Regularized FOCUSS (RFOCUSS) and being able to estimate two targets even with missing information up to 60%. Under the same conditions of missing information, IFOCUSS can estimate more targets with smaller positioning errors than RFOCUSS, achieving better convergence performance by eliminating almost all interference terms.

Large Environment Indoor Localization Leveraging Semi-Tensor Product Compression Sensing

The sparsity of the localization problem makes the Compression Sensing (CS) theory suitable for indoor localization in Wireless Local Area Networks (WLAN). However, in practice, we find that the location errors and computing complexity increase significantly as the dimensionality of the sparse vector and measurement matrix are high in a large environment, so most CS-based localization techniques are accompanied by coarse localization and AP selection stages. Therefore, in this paper, we first deduced the relationship between the number of Access Points (APs) and the dimensionality of the sparse vector theoretically to give the guideline that the number of sub-databases and APs should be obtained. Then an Adaptive Intuitionistic Fuzzy C-ordered Mean (AIFCOM) clustering is designed for the data with outliers in the environment with multipath effects. Finally, in the fine localization stage, we propose a Semi-tensor Product Compression Sensing (STP-CS) model to construct the measurement matrix, compared with the traditional CS model, our model not only remains more number of APs, but also decreases the dimensionality of measurement matrix, which can reduce the storage space and improve localization accuracy simultaneously.

Creating synthetic datasets for collaborative filtering recommender systems using generative adversarial networks

Research and education in machine learning requires diverse, representative, and open datasets that contain sufficient samples to handle the necessary training, validation, and testing tasks. Currently, the Recommender Systems area includes a large number of subfields in which accuracy and beyond-accuracy quality measures are continuously being improved. To feed this research variety, it is both necessary and convenient to reinforce the existing datasets with synthetic ones. This paper proposes a Generative Adversarial Network (GAN)-based method to generate collaborative filtering datasets in a parameterized way by selecting their preferred number of users, items, samples, and stochastic variability. This parameterization cannot be performed using regular GANs. Our GAN model is fed with dense, short, and continuous embedding representations of items and users, instead of sparse, large, and discrete vectors, to ensure fast and accurate learning, as compared to the traditional approach based on large and sparse input vectors. The proposed architecture includes a DeepMF model to extract the dense user and item embeddings and a clustering process to convert the dense GAN generated samples to the discrete and sparse samples necessary to create each required synthetic dataset. The results from three different source datasets show adequate distributions and expected quality values and evolutions in the generated datasets compared to the source datasets. Synthetic datasets and source codes are available to researchers.

Beyond Weisfeiler–Lehman with Local Ego-Network Encodings

Identifying similar network structures is key to capturing graph isomorphisms and learning representations that exploit structural information encoded in graph data. This work shows that ego networks can produce a structural encoding scheme for arbitrary graphs with greater expressivity than the Weisfeiler–Lehman (1-WL) test. We introduce IGEL, a preprocessing step to produce features that augment node representations by encoding ego networks into sparse vectors that enrich message passing (MP) graph neural networks (GNNs) beyond 1-WL expressivity. We formally describe the relation between IGEL and 1-WL, and characterize its expressive power and limitations. Experiments show that IGEL matches the empirical expressivity of state-of-the-art methods on isomorphism detection while improving performance on nine GNN architectures and six graph machine learning tasks.

A novel method for noise reduction of blade tip timing signals based on sparse representation and dictionary learning

Blade Tip Timing Technology (BTT) is a new type of blade vibration measurement method with non-contact measurement capabilities, which is associated with high efficiency and convenience. However, due to under-sampling feature of the BTT signal, the method can only measure part of the vibration information. Therefore, various BTT spectral reconstruction algorithms have been developed based on the under-sampling feature and sparsity of the blade vibration spectrum. Because of the harsh working environment of the blades, BTT signal typically contains high-level Gaussian noise in actual measurement, which significantly reduces frequency-domain sparsity and even aliases the vibration modals, resulting in a decrease in the performance of sparse reconstruction algorithms. In this paper, a multi-sequences BTT signal noise reduction method is proposed, which is based on sparse representation to suppress Gaussian noise. Through appropriately modifying the constraint in the ℓ0 optimization problem, the method filters out the noise elements in the sparse vector of the BTT signal. Then, denoising is completed by sparse inverse transformation. Finally, a global average-based singular value decomposition dictionary learning algorithm (GA-K-SVD) is proposed to generate an over-complete sparse dictionary which is adaptable to the original signal, to increase the effectiveness of GA-K-SVD. At last, simulation and experience are carried out to verify the performance optimization effect for sparse reconstruction algorithm and the effectiveness of proposed noise reduction method.

Adaptive Diffusion Pairwise Fused Lasso LMS Algorithm Over Networks.

The topic of identification for sparse vector in a distributed way has triggered great interest in the area of adaptive filtering. Grouping components in the sparse vector has been validated to be an efficient way for enhancing identification performance for sparse parameter. The technique of pairwise fused lasso, which can promote similarity between each possible pair of nonnegligible components in the sparse vector, does not require that the nonnegligible components have to be distributed in one or multiple clusters. In other words, the nonnegligible components may be randomly scattered in the unknown sparse vector. In this article, based on the technique of pairwise fused lasso, we propose the novel pairwise fused lasso diffusion least mean-square (PFL-DLMS) algorithm, to identify sparse vector. The objective function we construct consists of three terms, i.e., the mean-square error (MSE) term, the regularizing term promoting the sparsity of all components, and the regularizing term promoting the sparsity of difference between each pair of components in the unknown sparse vector. After investigating mean stability condition of mean-square behavior in theoretical analysis, we propose the strategy of variable regularizing coefficients to overcome the difficulty that the optimal regularizing coefficients are usually unknown. Finally, numerical experiments are conducted to verify the effectiveness of the PFL-DLMS algorithm in identifying and tracking sparse parameter vector.

Sparse Recovery of Finite Alphabet Signals Using AMP and Its Application in DoA Estimation

This study presents an approximate message passing (AMP)-based method for the reconstruction of sparse signals from multiple measurement vectors. Recently, Ziniel and Schniter have provided a general analysis of the AMP algorithm. They have assumed that the nonzero elements of the sparse signals have Gaussian distribution. This article extends their proposed method to the case that the nonzero elements of sparse vectors are chosen from a discrete finite alphabet. Therefore, the proposed algorithm is called approximate message passing with finite alphabet. Based on this assumption, the passing messages have been derived and the scheduling of the algorithm has been proposed using a three-step strategy. As an application, direction of arrival (DoA) estimation of digitally modulated signals is examined using simulations. The obtained results revealed that the proposed method would improve the DoA estimation accuracy compared to the conventional methods.

A Survey of Accelerating Parallel Sparse Linear Algebra

Sparse linear algebra includes the fundamental and important operations in various large-scale scientific computing and real-world applications. There exists performance bottleneck for sparse linear algebra since it mainly contains the memory-bound computations with low arithmetic intensity. How to improve its performance has increasingly become a focus of research efforts. Using parallel computing techniques to accelerate sparse linear algebra is currently the most popular method, while facing various challenges, e.g., large-scale data brings difficulties in storage, and the sparsity of data leads to irregular memory accesses and parallel load imbalance. Therefore, this article provides a comprehensive overview on acceleration of sparse linear algebra operations using parallel computing platforms, where we focus on four main classifications: sparse matrix-vector multiplication (SpMV), sparse matrix-sparse vector multiplication (SpMSpV), sparse general matrix-matrix multiplication (SpGEMM), and sparse tensor algebra. The takeaways from this article include the following: understanding the challenges of accelerating linear sparse algebra on various hardware platforms; understanding how structured data sparsity can improve storage efficiency; understanding how to optimize parallel load balance; understanding how to improve the efficiency of memory accesses; understanding how do the adaptive frameworks automatically select the optimal algorithms; and understanding recent design trends for acceleration of parallel sparse linear algebra.

Through the Wall Radar Imaging via Kronecker-structured Huber-type RPCA

The detection of multiple targets in an enclosed scene, from its outside, is a challenging topic of research addressed by Through-the-Wall Radar Imaging (TWRI). Traditionally, TWRI methods operate in two steps: first the removal of wall clutter then followed by the recovery of targets positions. Recent approaches manage in parallel the processing of the wall and targets via low rank plus sparse matrix decomposition and obtain better performances. In this paper, we reformulate this precisely via a RPCA-type problem, where the sparse vector appears in a Kronecker product. We extend this approach by adding a robust distance with flexible structure to handle heterogeneous noise and outliers, which may appear in TWRI measurements. The resolution is achieved via the Alternating Direction Method of Multipliers (ADMM) and variable splitting to decouple the constraints. The removal of the front wall is achieved via a closed-form proximal evaluation and the recovery of targets is possible via a tailored Majorization–Minimization (MM) step. The analysis and validation of our method is carried out using Finite-Difference Time-Domain (FDTD) simulated data, which show the advantage of our method in detection performance over complex scenarios.

Complex sparse Bayesian learning for guided wave dispersion curve estimation in plate-like structures

This paper proposes a new dispersion curve estimation method that employs the complex sparse Bayesian learning (CSBL). It is well accepted that guided wave packets are distorted because of the differences in propagation velocities at different frequencies; thus, the preceding velocity–frequency curve estimation is beneficial for wave packet recovery, feature recognition and defect localization. Conventional dispersion curve estimation methods, such as two-dimensional Fourier transform and multiple signal classification, are suitable for array signal and are restricted by the transducer aperture, leading to a small application scope. According to the frequency–response model of the guided wave, for each frequency, the responses obtained by the transducers can be sparsely represented based on an overcomplete dictionary matrix constructed using multiple discretized wavenumbers and known distances. A CSBL algorithm was developed to infer the posterior probability density function of the weight vector in the sparse representation. The non-zero elements in the sparse weight vector mean that the corresponding dictionary atoms indicated wavenumbers are contained in the frequency response, and the velocity–frequency curve can be finally derived from the wavenumber–frequency curve. The proposed CSBL method has a satisfactory capability to solve this sparse representation of the complex frequency response because a hierarchical form of the Laplace prior is employed to achieve a high degree of sparsity of the weight vector. This method effectively incorporates the real and imaginary parts of the complex frequency response by employing the same hyperparameter to integrate the known information. This method requires only a few randomly placed transducers; thus, it has a wide range of applications. The effectiveness of the proposed approach was validated using multiple guided-wave signals obtained through numerical simulations and an experimental study on a plate structure.

Efficient Compressed Sensing Reconstruction Algorithm for Nonnegative Vectors in Wireless Data Transmission

With the rapid development of 5G communication and wireless Internet of Things technology, the application of intelligent wearable devices based on wireless data transmission technology is becoming more and more popular. However, due to the bandwidth of wireless data transmission nodes and the power consumption of the device system, the use time and data storage of wearable devices are severely restricted. The compressed sensing (CS) technology has become an effective method to tackle this problem. CS technology includes data compressive sensing at the transmission end and data reconstruction at the receiving end. In this paper, we consider the reconstruction of nonnegative sparse vectors, an important problem in the area of CS for wireless data transmission. It is known that the interval-passing (IP) algorithm is a low-complexity message-passing type method for the problem. However, the reconstruction performance of the IP algorithm is inferior to that of the other state-of-the-art CS reconstruction algorithms such as the orthogonal matching pursuit (OMP) algorithm. In order to address the problem, we propose a two-stage reconstruction algorithm in this paper. The proposed algorithm applies the IP algorithm for the first stage of reconstruction. If the reconstruction fails, the OMP algorithm is then used on the basis of the results in the first stage. The proposed algorithm is evaluated and compared with other state-of-the-art algorithms by the probability of perfect reconstruction under the given sparsity order value. Simulation results suggest that the proposed two-stage algorithm can greatly improve the reconstruction performance of the IP algorithm and can even outperform the OMP algorithm. In addition, the low-complexity advantages of the IP algorithm are maintained in the proposed algorithm.

Sparse Bayesian Relevance Vector Machine Identification Modeling and Its Application to Ship Maneuvering Motion Prediction

In order to establish a sparse and accurate ship motion prediction model, a novel Bayesian probability prediction model based on relevance vector machine (RVM) was proposed for nonparametric modeling. The sparsity, effectiveness, and generalization of RVM were verified from two aspects: (1) the processed Sinc function dataset, and (2) the tank test dataset of the KRISO container ship (KCS) model. The KCS was taken as the main research plant, and the motion prediction models of KCS were obtained. The ε-support vector regression and υ-support vector regression were taken as the compared algorithms. The sparsity, effectiveness, and generalization of the three algorithms were analyzed. According to the trained prediction models of the three algorithms, the number of relevance vectors was compared with the number of support vectors. From the prediction results of the Sinc function and tank test datasets, the highest percentage of relevance vectors in the trained sample was below 17%. The final prediction results indicated that the proposed nonparametric models had good prediction performance. They could ensure good sparsity while ensuring high prediction accuracy. Compared with the SVR, the prediction accuracy can be improved by more than 14.04%, and the time consumption was also relatively lower. A training model with good sparsity can reduce prediction time. This is essential for the online prediction of ship motion.

A critical assessment of sparse PCA (research): why (one should acknowledge that) weights are not loadings

Principal component analysis (PCA) is an important tool for analyzing large collections of variables. It functions both as a pre-processing tool to summarize many variables into components and as a method to reveal structure in data. Different coefficients play a central role in these two uses. One focuses on the weights when the goal is summarization, while one inspects the loadings if the goal is to reveal structure. It is well known that the solutions to the two approaches can be found by singular value decomposition; weights, loadings, and right singular vectors are mathematically equivalent. What is often overlooked, is that they are no longer equivalent in the setting of sparse PCA methods which induce zeros either in the weights or the loadings. The lack of awareness for this difference has led to questionable research practices in sparse PCA. First, in simulation studies data is generated mostly based only on structures with sparse singular vectors or sparse loadings, neglecting the structure with sparse weights. Second, reported results represent local optima as the iterative routines are often initiated with the right singular vectors. In this paper we critically re-assess sparse PCA methods by also including data generating schemes characterized by sparse weights and different initialization strategies. The results show that relying on commonly used data generating models can lead to over-optimistic conclusions. They also highlight the impact of choice between sparse weights versus sparse loadings methods and the initialization strategies. The practical consequences of this choice are illustrated with empirical datasets.

A Globally Stable LPNN Model for Sparse Approximation.

The objective of compressive sampling is to determine a sparse vector from an observation vector. This brief describes an analog neural method to achieve the objective. Unlike previous analog neural models which either resort to the l1 -norm approximation or are with local convergence only, the proposed method avoids any approximation of the l1 -norm term and is probably capable of leading to the optimum solution. Moreover, its computational complexity is lower than that of the other three comparison analog models. Simulation results show that the error performance of the proposed model is comparable to several state-of-the-art digital algorithms and analog models and that its convergence is faster than that of the comparison analog neural models.

Multitarget Enumeration and Localization in Distributed MIMO Radar Based on Energy Modeling and Compressive Sensing

Target localization is one of the most important research topics in the field of radar signal processing. In this paper, the problem of multitarget enumeration and localization in the distributed multiple-input multiple-output (MIMO) radar with noncoherent processing mode is investigated. We first analyze the theoretical bound of the multitarget localization accuracy under the discrete time signal model and the Swerling 1 target model. It is determined by the Cram'r-Rao lower bound (CRLB) at low signal-to-noise ratio (SNR) and the sampling lower bound (SLB) when the SNR is high. Furthermore, an innovative multitarget enumeration and localization scheme is developed, which is based on the energy modeling of the multiple transmitter-receiver paths and the compressive sensing theory. To solve the sparse vector recovery issue, we design a lightweight iterative greedy pursuit algorithm including the similarity evaluation strategy. In addition, an iterative-based target position refinement process is designed to alleviate the off-grid problem caused by the spatial discretization. The proposal utilizes the samples of the raw signals and belongs to the category of the direct localization. Nevertheless, it has significantly higher computational efficiency and lower data communication burden than the conventional direct localization methods, while avoiding the complex data association encountered by the indirect localization methods. Finally, the simulation results validate the effectiveness and robustness of the proposed method.

A Duo Autoencoder-SVM based Approach for Secure Performance Monitoring of Industrial Conveyor Belt System

Process industries are fascinated by cyber-physical systems because of the potential to integrate physical systems and the cyber realm, resulting in efficient remote monitoring and control. The conveyor belt system has many critical parameters that require continuous attention, necessitating cyber-physical remote monitoring. Due to cloud-based monitoring of parameters, the system is vulnerable to cyber threats. The proposed technique combines a sparse autoencoder and support vector machine (SVM) to detect false data injection attacks (FDIAs) in the presence of sensor bias fault. The sparse autoencoder extracts sparse features and learns anomaly-free dynamics from the input sensor readings. Then, the trained SVM distinguishes attacks and fault by analysing reconstruction residuals of each measurement reading. The residuals also give an idea about the magnitude of abnormality. The proposed method's efficacy is evaluated in terms of accuracy, precision and false-alarm rate with the help of fault and FDIAs models.

Risk network of global energy markets

This study evaluates extreme uncertainty connectedness among top global energy firms. The sample comprises of 68 firms from four energy-related subsectors (oil & gas, oil & gas related equipment and services, multiline utilities, and renewable energy). To provide an overview of tail connectedness, we construct a high-dimensional network between firms by utilizing a generalized error decomposition and a sparse vector autoregression framework with a latent common factor. Our empirical results indicate that between the four subsectors, the renewable energy subsector exhibits the highest uncertainty transmission to other underlying subsectors, primarily credited to an increased within-subsector idiosyncratic uncertainty before the COVID-19 crisis. After the burst of the COVID-19 pandemic, due to the higher connectedness, the role of the renewable energy companies in the spillover network is further intensified. The uncertainty connectedness demonstrates a time-varying trait. While the oil and gas subsector exhibits greater long-term linkages with the oil and gas related equipment and services subsector, the long-run dynamics exhibit a lower interconnectedness as compared to the short-run. Finally, there is an increased connectedness among companies operating in the same subsector with similar size, attributing to similarity and competition.

Outage Analysis of Sparse Vector Coding-Based Downlink Multicarrier NOMA for URLLC

Ultra-reliable and low latency communication (uRLLC) is a promising use case in the fifth generation and beyond systems for critical and delay-sensitive applications with aid of short packet transmission. Nevertheless, multi-carrier non-orthogonal multiple access (MCNOMA) is a promising technique to achieve spectral efficiency and massive connectivity, however, inter-carrier interference (ICI) stands out to be a major downside. Recently, sparse vector coding (SVC) is proposed to fulfill the trade-off between reliability and latency, thereby achieving the benchmarks of uRLLC. In this paper, SVC based downlink MCNOMA (SVC-MCNOMA) system is considered and first it is shown that SVC-MCNOMA is free from ICI due to the sparse nature of the information. Further, compressed sensing-based recovery at the receiver end helps in reducing the latency as well as enhances reliability. Moreover, to study the system performance, this paper derives the closed form outage expressions of the SVC-MCNOMA system over the Rician fading channel and through Monte-Carlo simulation, the analytical results are validated. The analyses demonstrate that SVC-MCNOMA outperforms conventional MCNOMA (C-MCNOMA), thereby achieving spectral efficacy, ultra-reliability with low latency and eradicates the effect of ICI.

Machine learning models for predicting depression in Korean young employees.

The incidence of depression among employees has gradually risen. Previous studies have focused on predicting the risk of depression, but most studies were conducted using basic statistical methods. This study used machine learning algorithms to build models that detect and identify the important factors associated with depression in the workplace. A total of 503 employees completed an online survey that included questionnaires on general characteristics, physical health, job-related factors, psychosocial protective, and risk factors in the workplace. The dataset contained 27 predictor variables and one dependent variable which referred to the status of employees (normal or at the risk of depression). The prediction accuracy of three machine learning models using sparse logistic regression, support vector machine, and random forest was compared with the accuracy, precision, sensitivity, specificity, and AUC. Additionally, the important factors identified via sparse logistic regression and random forest. All machine learning models demonstrated similar results, with the lowest accuracy obtained from sparse logistic regression and support vector machine (86.8%) and the highest accuracy from random forest (88.7%). The important factors identified in this study were gender, physical health, job, psychosocial protective factors, and psychosocial risk and protective factors in the workplace. The results of this study indicated the potential of machine learning models to accurately predict the risk of depression among employees. The identified factors that influence the risk of depression can contribute to the development of intelligent mental healthcare systems that can detect early signs of depressive symptoms in the workplace.