Sparsity Information Research Articles

Additive Functional Cox Model

We propose the additive functional Cox model to flexibly quantify the association between functional covariates and time to event data. The model extends the linear functional proportional hazards model by allowing the association between the functional covariate and log hazard to vary nonlinearly in both the functional domain and the value of the functional covariate. Additionally, we introduce critical transformations of the functional covariate which address the weak model identifiability in areas of information sparsity and discuss their impact on interpretation and inference. We also introduce a novel estimation procedure that accounts for identifiability constraints directly during model fitting. Methods are applied to the National Health and Nutrition Examination Survey 2003–2006 accelerometry data and quantify new and interpretable circadian patterns of physical activity that are associated with all-cause mortality. We also introduce a simple and novel simulation framework for generating survival data with functional predictors which resemble the observed data. The accompanying inferential R software is fast, open source, and publicly available. Our data application and simulations are fully reproducible through the accompanying vignette. Supplementary materials for this article are available online.

Adaptively-Regularized Compressive Sensing With Sparsity Bound Learning

In this letter, an adaptively-regularized iterative reweighted least squares algorithm with sparsity bound learning is designed to efficiently recover sparse signals from measurements. In particular, at each iteration the support of estimated signal is exploited to construct a sparsity-promoting matrix and, then, formulate an adaptive regularization. Since this algorithm could learn sparsity information at each iteration, it ensures a sparser and sparser solution, and the mean squared error analysis corroborates its convergence. Experimental results demonstrate that the proposed algorithm outperforms other typical ones in terms of sparsity level, compressive ratio, and detection probability.

Enhanced Sparsity Prior Model for Low-Rank Tensor Completion.

Conventional tensor completion (TC) methods generally assume that the sparsity of tensor-valued data lies in the global subspace. The so-called global sparsity prior is measured by the tensor nuclear norm. Such assumption is not reliable in recovering low-rank (LR) tensor data, especially when considerable elements of data are missing. To mitigate this weakness, this article presents an enhanced sparsity prior model for LRTC using both local and global sparsity information in a latent LR tensor. In specific, we adopt a doubly weighted strategy for nuclear norm along each mode to characterize global sparsity prior of tensor. Different from traditional tensor-based local sparsity description, the proposed factor gradient sparsity prior in the Tucker decomposition model describes the underlying subspace local smoothness in real-world tensor objects, which simultaneously characterizes local piecewise structure over all dimensions. Moreover, there is no need to minimize the rank of a tensor for the proposed local sparsity prior. Extensive experiments on synthetic data, real-world hyperspectral images, and face modeling data demonstrate that the proposed model outperforms state-of-the-art techniques in terms of prediction capability and efficiency.

Convergence of proximal algorithms with stepsize controls for non-linear inverse problems and application to sparse non-negative matrix factorization

We consider a general ill-posed inverse problem in a Hilbert space setting by minimizing a misfit functional coupling with a multi-penalty regularization for stabilization. For solving this minimization problem, we investigate two proximal algorithms with stepsize controls: a proximal fixed point algorithm and an alternating proximal algorithm. We prove the decrease of the objective functional and the convergence of both update schemes to a stationary point under mild conditions on the stepsizes. These algorithms are then applied to the sparse and non-negative matrix factorization problems. Based on a priori information of non-negativity and sparsity of the exact solution, the problem is regularized by corresponding terms. In both cases, the implementation of our proposed algorithms is straight-forward since the evaluation of the proximal operators in these problems can be done explicitly. Finally, we test the proposed algorithms for the non-negative sparse matrix factorization problem with both simulated and real-world data and discuss reconstruction performance, convergence, as well as achieved sparsity.

The Sprawlter Graph Readability Metric: Combining Sprawl and Area-Aware Clutter.

Graph drawing readability metrics are routinely used to assess and create node-link layouts of network data. Existing readability metrics fall short in three ways. The many count-based metrics such as edge-edge or node-edge crossings simply provide integer counts, missing the opportunity to quantify the amount of overlap between items, which may vary in size, at a more fine-grained level. Current metrics focus solely on single-level topological structure, ignoring the possibility of multi-level structure such as large and thus highly salient metanodes. Most current metrics focus on the measurement of clutter in the form of crossings and overlaps, and do not take into account the trade-off between the clutter and the information sparsity of the drawing, which we refer to as sprawl. We propose an area-aware approach to clutter metrics that tracks the extent of geometric overlaps between node-node, node-edge, and edge-edge pairs in detail. It handles variable-size nodes and explicitly treats metanodes and leaf nodes uniformly. We call the combination of a sprawl metric and an area-aware clutter metric a sprawlter metric. We present an instantiation of the sprawlter metrics featuring a formal and thorough discussion of the crucial component, the penalty mapping function. We implement and validate our proposed metrics with extensive computational analysis of graph layouts, considering four layout algorithms and 56 layouts encompassing both real-world data and synthetic examples illustrating specific configurations of interest.

OSRanP: A Novel Way for Radar Imaging Utilizing Joint Sparsity and Low-Rankness

Synthetic aperture radar (SAR) has extensive applications in both civilian and military fields for its ability to create high-resolution images of the ground target without being affected by weather conditions and daytime or nighttime effect. As fueled by the several decades’ advancement of SAR, existing SAR systems exhibit high imaging capabilities, namely, significantly high two-dimensional resolution. However, in accordance with Nyquist's sampling theory, the increase in resolution implies an evident increase in the amount of sampling data, thereby causing numerous limitations to practical application. To solve this problem, several compressive sensing (CS) and matrix sensing (MS) techniques have been applied to SAR imaging, wherein the existing knowledge of sparsity or low-rankness is exploited to reconstruct the SAR image based on an under-sampled SAR raw data. In this study, we take a different approach, wherein redundancy property of the SAR image is further exploited. The SAR image is split into a sparse matrix and a low rank matrix. Thus, the SAR imaging processor is modelled as a problem of joint sparse and low rank matrices recovery. An Orthogonal Sparse and Rank-one Pursuit (OSRanP) algorithm is newly proposed to solve this problem in SAR imaging case, where there isn’t any prior information of the exact sparsity or low-rankness value at hand. As revealed from the results here, the proposed method outmatches the CS and MS methods in sampling efficiency in both simulations and experiments.

SPRITE: 3-D SParse Radar Imaging TEchnique

An original 3-D high resolution radar imaging approach, called SPRITE for “SParse Radar Imaging TEchnique,” is presented. It incorporates in an original way the available prior knowledge about the electromagnetic backscattering, extending the commonly used sparse point-wise scatterers to sparse facet-wise scatterers. It is based on a regularization scheme that accounts for information of sparsity and support. The radar map formation is performed efficiently based on a penalized and constrained criterion, and an Alternating Direction Method of Multipliers algorithm, largely used in image restoration and machine learning. It is customized in a such way that, at each iteration, the map update is fast in the frequency domain by 3-D FFT and IFFT while the updates of the auxiliary variables are direct and separable. SPRITE is both evaluated on synthetic and real measurement data from a spherical measurement setup. In comparison to the conventional method, called Polar Format Algorithm, the resolution is drastically enhanced. The main scatterers are recovered with increased accuracy, leading to a deeper understanding of the scattering behavior. Furthermore, compared to recent $\ell _1$ based methods that are limited to point-wise scatterers, SPRITE and its facet-wise scatterers provide an impressive and improved spatial backscattering representation.

A Grant-Free Method for Massive Machine-Type Communication With Backward Activity Level Estimation

Massive machine type communications (mMTC) is one of the three major scenarios of the fifth generation (5G) communication system, and raises new challenges for the development of new radio access technology. Unlike human type communications (HTC), mMTC is typically characterised by a massive number of devices, small-sized packets, low or no mobility, low energy consumption and sporadic transmission, which requires novel solutions. In this paper, we propose the 2-step random access with early data transmission (2-step-EDT) framework. To solve the optimization problem proposed in the framework, we introduce an algorithm, namely, Backward Sparsity Adaptive Matching Pursuit with Checking and Projecting (BSAMP-CP), which jointly conducts the sparsity level estimation, active device detection, channel estimation and data recovery in two phases. Specifically, in the first phase, BSAMP-CP conducts the sparsity level estimation in a backward manner exploiting the data length diversity information. In the second phase, BSAMP-CP jointly conducts activity detection, channel estimation and data recovery, taking the joint sparsity information of pilot and data symbols, the error checking information and the modulation constellation information into account. Furthermore, we provide a theoretical analysis on the convergence of the proposed BSAMP-CP in the noiseless case and the rationale for the improvement yielded by exploiting data length diversity. Simulation results demonstrate the superiority of the proposed solution in comparison to other existing methods.

Headspace Oxygen Concentration Measurement for Pharmaceutical Glass Bottles in Open-Path Optical Environment Using TDLAS/WMS

Accurate measurement of oxygen concentration for pharmaceutical glass bottles is of great significance to ensure the asepsis of medicine and stability of ingredients. With merits of high sensitivity, low cost, noncontact, and real-time response, the wavelength-modulation-based tunable diode laser absorption spectroscopy (TDLAS/WMS) technology shows great potential to achieve in-site oxygen concentration detection by the single-line spectrum measurement. This article focuses on headspace oxygen concentration measurement in the open-path optical environment, which is extremely challenging due to the short light path length and random ambient noises. First, a signal reconstruction method is established based on the discrete wavelet packet transform (DWPT), where random noise suppression is implicitly achieved. Then, oxygen concentration is inversed among the data between two adjacent valley values of the demodulated second-harmonic signal by multiple linear regression (MLR), and the linear discriminant analysis (LDA) is imported to enhance the information sparsity of second-harmonic signal matrix and to address the multicollinearity problem. Simulation results prove that our proposed method achieved considerable detection accuracy with the average absolute error of 0.05%. This article also designed a TDLAS/WMS prototype, and the experimental results aiming at glass bottles with different oxygen concentration of 0%, 5%, 10%, and 21% in open-path optical environment indicate our method has achieved an encouraging average absolute error of 0.54% and can survive well when the normalized signal-to-noise ratio (SNR) is within 0.85–1. These results promise that the proposed methodology can be widely applied in in-site automatic optic inspection (AOI) instrumentation of headspace oxygen concentration measurement for glass vials.

Discovering influential factors in variational autoencoders

In the field of machine learning, it is still a critical issue to identify and supervise the learned representation without manually intervening or intuition assistance to extract useful knowledge or serve for the downstream tasks. In this work, we focus on supervising the influential factors extracted by the variational autoencoder (VAE). The VAE is proposed to learn independent low dimension representation while facing the problem that sometimes pre-set factors are ignored. We argue that the mutual information of the input and each learned factor of the representation plays a necessary indicator of discovering the influential factors. We find the VAE objective inclines to induce mutual information sparsity in factor dimension over the data intrinsic dimension and therefore result in some non-influential factors whose function on data reconstruction could be ignored. We show mutual information also influences the lower bound of VAE’s reconstruction error and downstream classification task. To make such indicator applicable, we design an algorithm for calculating the mutual information for VAE and prove its consistency. Experimental results on MNIST, CelebA and DEAP datasets show that mutual information can help determine influential factors, of which some are interpretable and can be used to further generation and classification tasks, and help discover the variant that connects with emotion on DEAP dataset.

Resilient Reinforcement in Secure State Estimation Against Sensor Attacks With A Priori Information

Recent control systems severe depend on information technology infrastructures, especially the Internet of things (IoT) devices, which create many opportunities for the interaction between the physical world and cyberspace. Due to the tight connection, however, cyber attacks have the potential to generate evil consequences for the physical entities, and therefore, securing control systems is a vital issue to be addressed for building smart societies. To this end, this paper especially deals with the state estimation problem in the presence of malicious sensor attacks. Unlike the existing work, in this paper, we consider the problem with a priori information of the state to be estimated. Specifically, we address three prior knowledge—the sparsity information, $ (\alpha, \bar{n}_0)$ -sparsity information, and side information, and in each scenario, we show that the state can be reconstructed even if more sensors are compromised. This implies that the prior information reinforces the system resilience against malicious sensor attacks. Then, an estimator under sensor attacks considering the information is developed and, under a certain condition, the estimator can be relaxed into a tractable convex optimization problem. Further, we extend this analysis to systems in the presence of measurement noises, and it is shown that the prior information reduces the state-estimation error caused by the noise. The numerical simulations in a diffusion process finally illustrate the reinforcement and error-reduction results with the information.

Exploring spatial and channel contribution for object based image retrieval

With the rapid development of deep learning methods, researchers have gradually shifted the research focus from hand-crafted features to deep features in the field of the content-based image retrieval (CBIR). A great deal of attention has been paid to aggregate the extracted features from the convolutional layer in the deep convolutional neural network (CNN) into a global representation vector for CBIR. In this paper, we propose a simple but effective method which called Strong-Response-Stack-Contribution (SRSC) to generate the global representation vector for object retrieval. As we know, for object retrieval, when using CNN to extract features, what we want is to extract features in the region of interest (ROI). So we explored spatial and channel contribution to help us focus more on ROI and make the global image representation vector more representative. The process of the approach SRSC is to first generate spatial contribution according to the degree of channel response intensity. Then, we generate channel contribution by joining the sparsity information and the element-value information together. Finally, the global representation vector is generated according to spatial and channel contribution to perform image retrieval. Experiments on Oxford and Paris buildings datasets show the effectiveness of the proposed approach.

GAP: A General Framework for Information Pooling in Two-Sample Sparse Inference

This article develops a general framework for exploiting the sparsity information in two-sample multiple testing problems. We propose to first construct a covariate sequence, in addition to the usual primary test statistics, to capture the sparsity structure, and then incorporate the auxiliary covariates in inference via a three-step algorithm consisting of grouping, adjusting and pooling (GAP). The GAP procedure provides a simple and effective framework for information pooling. An important advantage of GAP is its capability of handling various dependence structures such as those arise from high-dimensional linear regression, differential correlation analysis, and differential network analysis. We establish general conditions under which GAP is asymptotically valid for false discovery rate control, and show that these conditions are fulfilled in a range of settings, including testing multivariate normal means, high-dimensional linear regression, differential covariance or correlation matrices, and Gaussian graphical models. Numerical results demonstrate that existing methods can be significantly improved by the proposed framework. The GAP procedure is illustrated using a breast cancer study for identifying gene–gene interactions.

Novel Adaptive Distributed Compressed Sensing Algorithm for Estimating Channels in Doubly-Selective Fading OFDM Systems

Doubly-selective (DS) fading channel is often occurred in many orthogonal frequency division multiplexing (OFDM) communication systems, such as high-speed rail communication systems and underwater acoustic (UWA) wireless networks. It is challenging to provide an accurate and fast estimation over the doubly-selective channel, due to the strong Doppler shift. This paper addresses the doubly selective channel estimation problem based on complex exponential basis expansion model (CE-BEM) in OFDM systems from the perspective of distributed compressive sensing (DCS). We propose a novel DCS-based improved sparsity adaptive matching pursuit (DCS-IMSAMP) algorithm. The advantage of the proposed algorithm is that it can exploit the joint channel sparsity information using dynamic threshold, variable step size and tailoring mechanism. Simulation results show that the proposed algorithm achieves 5dB performance gain with faster operation speed, in comparison with traditional DCS-based sparsity adaptive matching pursuit (DCS-SAMP) algorithm.

Sparse representation of classified patches for CS-MRI reconstruction

Compressed sensing MRI (CS-MRI) has demonstrated its potential in image reconstruction from undersampling k-space data to accelerate scanning time. Exact CS-MRI reconstruction is based on the image prior information of sparsity, and therefore, it relies on two important aspects of sparse domain and sparse coding. In this work, a patch based uniform model according to orthogonal dictionary learning and lp norm minimization (ODNM) is developed. First, image patches are classified into multi-classes by a use of a modified clustering method and the corresponding orthogonal dictionary is learned for each class. In addition, the non-convex lp norm regularization is employed to promote the sparsity of patch coefficients. In order to solve the proposed reconstruction model, the alternating direction method (ADM) steps are developed in which orthogonal dictionary learning, non-convex sparse coding, and image reconstruction are simultaneously conducted. In the special case of p=0.5, a fast shrinkage operator is proposed to reduce the computational complexity. Extensive experiments on real complex valued MR images under various sampling patterns demonstrate that the proposed method achieves a better performance than several state-of-the-art algorithms.

Underwater Acoustic OFDM Channel Estimation with Unknown Sparsity

Underwater acoustic channels estimation accuracy seriously affects the demodulation performance of the receiver in underwater acoustic (UWA) orthogonal frequency division multiplexing (OFDM) communications. Many channel estimation algorithms based on compressed sensing (CS) have been proposed. However, these algorithms usually needs the information of sparsity, while the information of sparsity is unavailable in many UWA OFDM communications. In this paper, a novel channel estimation algorithm based on fractional Fourier transform (FRFT) and orthogonal matching pursuit (OMP) is proposed, FRFT is introduced to process the synchronization signal, so sparsity is estimated. OMP is adopted to reconstruct channel impulse response (CIR). The most innovative feature of the proposed algorithm is the estimation of sparsity, which makes OMP algorithm more practical. Simulation results show that the proposed algorithm outperforms the least square (LS) and in UWA OFDM communication.

A Fuzzy Selection Compressive Sampling Matching Pursuit Algorithm for its Practical Application

In compressive sampling matching pursuit algorithm, it requires that the sparsity information of original signal to control the size of the preliminary atomic set and the maximum number of the algorithm iteration. This weakens the reconstruction accuracy, increases the computation complexity and limits its practical application capacity. To overcome the problem, an improved method is proposed. The proposed method firstly sets a fixed step-size as the assumed sparsity to expand the preliminary atomic set at the initial stage when the sparsity information is unknown. Secondly, the proposed algorithm adopts the fuzzy threshold strategy to select the more relevant atoms from the preliminary atomic set to expand the candidate atomic set. Finally, the double threshold control method, multiply stages setting and variable step-size method are used to control the iteration stop condition and adjust the estimated sparsity. When the two threshold iteration stop conditions are simultaneously satisfied, the iteration stops, which shows that the reconstructed signal better approximated the original signal, and the reconstruction performance is the best. Otherwise, if only one of the conditions is satisfied, the size of the estimated sparsity is increased by the variable step size method to reduce the error between the reconstructed signal and original signal. In addition, we extended the proposed algorithm to the multiple measurement vectors scenario for joint sparse signal recovery. Simulation results indicate that the proposed algorithm is better than the other method in terms of the reconstruction performance in single measurement vector and multiple measurement vector cases.

A new adaptive-weighted total variation sparse-view computed tomography image reconstruction with local improved gradient information.

Inspired by the compressed sensing (CS) theory, introducing priori information of sparse image into sparse-view reconstruction algorithm of computed tomography (CT) can improve image quality. In recent years, as a special case of CS, total variation (TV) reconstruction algorithm that uses both image sparsity and prior information of edge direction have attracted much research interest in sparse-view image reconstruction due to its ability to preserve image edges. In this paper, we propose a new adaptive-weighted total variation (NAWTV) algorithm for CT image reconstruction, which is derived by considering local gradient direction continuity and the anisotropic edge property. The anisotropic edge property is used to consolidate the image sparsity, where the associated weights are expressed as a combination of exponential and cosine function. The weights can also be adjusted adaptively according the local image intensity gradient. The NAWTV algorithm is numerically implemented with gradient descent method. The typical Shepp-Logan phantom and FORBILD head phantom are employed to perform image reconstruction simulation. To evaluate performance of NAWTV algorithm, we compared it with TV and AwTV reconstruction algorithms in experiments. Numerical experimental results verified the effectiveness and feasibility of the proposed algorithm. Comparison results also showed that the NAWTV algorithm achieved a satisfactory performance in suppressing artifacts and preserving the edge structure details information of the reconstructed image.

An Improved Algorithm based on MOLS for CS

The purpose of sparse recovery based on compressed sensing is to reconstruct sparse signals from linear compressed measurements. Greedy algorithm is often used to solve the inverse problem of underdetermined equations. Both Orthogonal Matching Pursuit(OMP) and Orthogonal Least Squares(OLS) greedy algorithms have been widely applied. Unlike the OMP algorithm, the first task of the OLS algorithm is to find the support set dropping the residual fastest. Multiple Orthogonal Least Squares(MOLS) algorithm adds the idea of multiple support set selection to the OLS algorithm, which greatly reduces the time complexity of the OLS algorithm, but needs to take the sparsity K as a priori condition. Based on this, a novel sparse recovery algorithm called Changing Stage Orthogonal Least Squares(CSOLS) is proposed in this paper. Compared with the MOLS algorithm, the most innovative feature of the CSOLS algorithm is the signal reconstruction ability without a priori information sparsity, finishing sparse recovery by conditionally broadening the search step. Compared with the MOLS algorithm and the traditional greedy algorithms with regard to the Frequency of Exact Reconstruction(FER) under the different sparsity and measurements, the CSOLS algorithm shows terrific recovery performance.

Compressive sensing-based vibration signal reconstruction using sparsity adaptive subspace pursuit

Wireless sensors produce large amounts of data in long-term online monitoring following the Shannon–Nyquist theorem, leading to a heavy burden on wireless communications and data storage. To address this problem, compressive sensing which allows wireless sensors to sample at a much lower rate than the Nyquist frequency has been considered. However, the lower rate sacrifices the integrity of the signal. Therefore, reconstruction from low-dimension measurement samples is necessary. Generally, the reconstruction needs the information of signal sparsity in advance, whereas it is usually unknown in practical applications. To address this issue, a sparsity adaptive subspace pursuit compressive sensing algorithm is deployed in this article. In order to balance the computational speed and estimation accuracy, a half-fold sparsity estimation method is proposed. To verify the effectiveness of this algorithm, several simulation tests were performed. First, the feasibility of subspace pursuit algorithm is verified using random sparse signals with five different sparsities. Second, the synthesized vibration signals for four different compression rates are reconstructed. The corresponding reconstruction correlation coefficient and root mean square error are demonstrated. The high correlation and low error result mean that the proposed algorithm can be applied in the vibration signal process. Third, implementation of the proposed approach for a practical vibration signal from an offshore structure is carried out. To reduce the effect of signal noise, the wavelet de-noising technique is used. Considering the randomness of the sampling, many reconstruction tests were carried out. Finally, to validate the reliability of the reconstructed signal, the structure modal parameters are calculated by the Eigensystem realization algorithm, and the result is only slightly different between original and reconstructed signal, which means that the proposed method can successfully save the modal information of vibration signals.