Sparse Modeling Approach Research Articles

Sparse modeling for climate variable selection across trophic levels.

Understanding how populations respond to climate is fundamentally important to many questions in ecology, evolution, and conservation biology. Climate is complex and multifaceted, with aspects affecting populations in different and sometimes unexpected ways. Thus, when measuring the changing climate it is important to consider the complexity of the phenomenon and the number of ways it can be characterized through different metrics. We used a Bayesian sparse modeling approach to select among 80 metrics of climate and applied the approach to 19 datasets of bird, insect, and plant population responses to abiotic conditions as case studies of how the method can be applied for climate variable selection in a time series context. For phenological datasets, mean spring temperature was frequently selected as an important climate driver, while selected predictors were more diverse for population metrics such as abundance or reproductive success. The climate variable selection approach presented here can help to identify potential climate metrics when there is limited physiological or mechanistic information to make an a priori variable selection, and is broadly applicable across studies on population responses to climate.

Sparse Signal Models for Data Augmentation in Deep Learning ATR

Automatic target recognition (ATR) algorithms are used to classify a given synthetic aperture radar (SAR) image into one of the known target classes by using the information gleaned from a set of training images that are available for each class. Recently, deep learning methods have been shown to achieve state-of-the-art classification accuracy if abundant training data are available, especially if they are sampled uniformly over the classes and in their poses. In this paper, we consider the ATR problem when a limited set of training images are available. We propose a data-augmentation approach to incorporate SAR domain knowledge and improve the generalization power of a data-intensive learning algorithm, such as a convolutional neural network (CNN). The proposed data-augmentation method employs a physics-inspired limited-persistence sparse modeling approach, which capitalizes on the commonly observed characteristics of wide-angle synthetic aperture radar (SAR) imagery. Specifically, we fit over-parametrized models of scattering to limited training data, and use the estimated models to synthesize new images at poses and sub-pixel translations that are not available in the given data in order to augment the limited training data. We exploit the sparsity of the scattering centers in the spatial domain and the smoothly varying structure of the scattering coefficients in the azimuthal domain to solve the ill-posed problem of the over-parametrized model fitting. The experimental results show that, for the training on the data-starved regions, the proposed method provides significant gains in the resulting ATR algorithm’s generalization performance.

Sparse Modeling Approach for Quasiclassical Theory of Superconductivity

We propose the sparse modeling approach for quasiclassical theory of superconductivity, which reduces the computational cost of solving the gap equations. The recently proposed sparse modeling approach is a consequence of the Green’s function having less information than its spectral function and hence is compressible without loss of relevant information. Using the intermediate representation of the Green’s function in the sparse modeling approach, the gap equation can be solved with only 10–100 sampled Matsubara Green’s functions, whereas the conventional quasiclassical theory requires 100–1000. The efficiency of the proposed method is demonstrated in bulk and vortex states, through self-consistently solve of the Eilenberger and gap equations. Consequently, the sparse modeling approach is concluded to be appropriate for all theoretical methods based on the Matsubara formalism in the quasiclassical theory of superconductivity.

Bayesian feature selection for radiomics using reliability metrics.

Introduction: Imaging of tumors is a standard step in diagnosing cancer and making subsequent treatment decisions. The field of radiomics aims to develop imaging based biomarkers using methods rooted in artificial intelligence applied to medical imaging. However, a challenging aspect of developing predictive models for clinical use is that many quantitative features derived from image data exhibit instability or lack of reproducibility across different imaging systems or image-processing pipelines. Methods: To address this challenge, we propose a Bayesian sparse modeling approach for image classification based on radiomic features, where the inclusion of more reliable features is favored via a probit prior formulation. Results: We verify through simulation studies that this approach can improve feature selection and prediction given correct prior information. Finally, we illustrate the method with an application to the classification of head and neck cancer patients by human papillomavirus status, using as our prior information a reliability metric quantifying feature stability across different imaging systems.

Extracting the geochemical characteristics of magmas in different global tectono-magmatic settings using sparse modeling

In this study, key geochemical features of magmas formed in eight different tectono-magmatic settings (mid-ocean ridges, oceanic islands, oceanic plateaus, continental flood basalt provinces, intra-oceanic arcs, continental arcs, island arcs, and back-arc basins) are presented that were obtained using a machine-learning-based statistical model. We analyzed geochemical data for volcanic rocks compiled from the global geochemical databases based on statistical model fitting. We used the sparse modeling approach, with which we can objectively identify a small number of fundamental features from a large number of observations. This approach allowed us to identify a small number of representative geochemical features from a total of 857 variables, including major and trace element concentrations, isotope ratios, and all possible ratios and multiplications of elements. Based on the statistical analysis, we present a small number (2–4) of key geochemical features for each tectono-magmatic setting. The extracted geochemical features and associated diagrams can be used to examine geochemical similarities and differences between tectono-magmatic settings and to identify the geochemical characteristics of unknown samples. Based on the extracted geochemical characteristics, we discuss the processes that may lead to the formation of magmas in different tectono-magmatic settings. Our statistical analysis shows that the geochemical signatures of magmas vary with the tectono-magmatic setting, as do the geochemical processes involved in magma generation.

Sparse and Low-Rank Optimization for Pliable Index Coding via Alternating Projection

Pliable index coding (PICOD) has recently been regarded as a promising solution that exploits the coding advantage to improve communication efficiency of content-type systems (e.g., recommendation system), where clients are pliable and are interested in receiving any new message that they do not have. PICOD aims to find an effective coding strategy that satisfies the demands of all clients with the minimum number of transmissions. However, most of the previous works mainly provided theoretical understanding on PICOD in special instances based on greedy algorithms. In contrast, in this paper, we present a flexible sparse and low-rank matrix modeling approach to minimize the number of transmissions for the general PICOD problems. This is achieved by establishing generalized pliable alignment conditions to guarantee the requirements of all clients. As the resulting non-convex problem is highly intractable, we further develop an alternating pursuit framework to detect the rank of the matrix to be recovered by using the rank-increasing strategy. To address the feasibility-detection issues in the existing methods, we propose an alternating projection algorithm, which admits closed-form expressions and avoids excessive sparsity inducing. Moreover, we establish the global convergence of the alternating projection algorithm with random initial points. Simulation results demonstrate that the proposed alternating pursuit algorithm significantly reduces the number of transmissions compared to the state-of-the-art methods.

Robust analytic continuation combining the advantages of the sparse modeling approach and the Padé approximation

Analytic continuation (AC) from the imaginary-time Green's function to the spectral function is a crucial process for numerical studies of the dynamical properties of quantum many-body systems. This process, however, is an ill-posed problem; that is, the obtained spectrum is unstable against the noise of the Green's function. Though several numerical methods have been developed, each of them has its own advantages and disadvantages. The sparse modeling (SpM) AC method, for example, is robust against the noise of the Green's function but suffers from unphysical oscillations in the low-energy region. We propose a new method that combines the SpM AC with the Pad\'{e} approximation. This combination, called SpM-Pad\'{e}, inherits robustness against noise from SpM and low-energy accuracy from Pad\'{e}, compensating for the disadvantages of each. We demonstrate that the SpM- Pad\'{e} method yields low-variance and low-biased results with almost the same computational cost as that of the SpM method.

Recovery of Blood Flow From Undersampled Photoacoustic Microscopy Data Using Sparse Modeling.

Photoacoustic microscopy (PAM) leverages the optical absorption contrast of blood hemoglobin for high-resolution, multi-parametric imaging of the microvasculature in vivo. However, to quantify the blood flow speed, dense spatial sampling is required to assess blood flow-induced loss of correlation of sequentially acquired A-line signals, resulting in increased laser pulse repetition rate and consequently optical fluence. To address this issue, we have developed a sparse modeling approach for blood flow quantification based on downsampled PAM data. Evaluation of its performance both in vitro and in vivo shows that this sparse modeling method can accurately recover the substantially downsampled data (up to 8 times) for correlation-based blood flow analysis, with a relative error of 12.7 ± 6.1 % across 10 datasets in vitro and 12.7 ± 12.1 % in vivo for data downsampled 8 times. Reconstruction with the proposed method is on par with recovery using compressive sensing, which exhibits an error of 12.0 ± 7.9 % in vitro and 33.86 ± 26.18 % in vivo for data downsampled 8 times. Both methods outperform bicubic interpolation, which shows an error of 15.95 ± 9.85 % in vitro and 110.7 ± 87.1 % in vivo for data downsampled 8 times.

Finding an optimal dictionary of different wavelet types using sparse modeling to denoise ECG signal

Abstract Sparse signal modeling often reconstructs a signal with few atoms from a pre-defined dictionary. Hence the choice of wavelet dictionary that represents the sparsity of the target signal is crucial in sparse modeling approach. The challenge of finding an optimal dictionary of different wavelet types using sparse denoising model (SDM) to denoise ECG signal is investigated in this work. A method of finding an optimal wavelet dictionary from a set of orthogonal wavelet sub-dictionaries by the means of the best correlation with ECG signal, is developed. The highly correlated sub-dictionaries from three wavelet dictionaries, namely daubechies, symlets, coiflets and discrete cosine transform are combined to construct an overcomplete dictionary. The weight of Akaike’s information criterion and the signal-to-noise ratio improvement are considered as a criterion to evaluate the performance of the proposed SDM. The results indicate that multi-wavelet dictionary of different types is highly sparse and efficient in denoising the target signal, e.g., ECG.

Data-Driven Analysis of Nonlinear Heterogeneous Reactions through Sparse Modeling and Bayesian Statistical Approaches.

Heterogeneous reactions are chemical reactions that occur at the interfaces of multiple phases, and often show a nonlinear dynamical behavior due to the effect of the time-variant surface area with complex reaction mechanisms. It is important to specify the kinetics of heterogeneous reactions in order to elucidate the microscopic elementary processes and predict the macroscopic future evolution of the system. In this study, we propose a data-driven method based on a sparse modeling algorithm and sequential Monte Carlo algorithm for simultaneously extracting substantial reaction terms and surface models from a number of candidates by using partial observation data. We introduce a sparse modeling approach with non-uniform sparsity levels in order to accurately estimate rate constants, and the sequential Monte Carlo algorithm is employed to estimate time courses of multi-dimensional hidden variables. The results estimated using the proposed method show that the rate constants of dissolution and precipitation reactions that are typical examples of surface heterogeneous reactions, necessary surface models, and reaction terms underlying observable data were successfully estimated from only observable temporal changes in the concentration of the dissolved intermediate products.

Sparse modeling and monitoring for industrial processes using sparse, distributed principal component analysis

Driven by the strong demand for sparsity in dimensional reduction techniques, a sparse modeling and monitoring approach based on sparse, distributed principal component analysis (SDPCA) is proposed to achieve sparsity. To this end, the data set is first divided into highly correlated blocks (HCBs) and one remainder block (RB) on the basis of the mutual-information-based correlation matrix. From this, the sparse loading vectors for the HCBs are obtained using the PCA models, while for the RB, it is obtained using the sparse PCA model. It is worth noting that the sparsity in SDPCA enables the sparse loading vectors to produce interpretable principal components, which keeps the correlations between the highly correlated variables and achieves the sparsity for the weakly correlated ones. Moreover, to fully appreciate the interpretation of the sparse principal components, a fault diagnosis strategy named blockwise contribution plots is proposed by first determining the faulty block, and then, identifying the faulty variables. Compared with PCA and SPCA, the proposed SDPCA detects more faulty samples and gives more accurate diagnosis results.

Sparse modeling approach to obtaining the shear viscosity from smeared correlation functions

We propose the sparse modeling method to estimate the spectral function from the smeared correlation functions. We give a description of how to obtain the shear viscosity from the correlation function of the renormalized energy-momentum tensor (EMT) measured by the gradient flow method (C (t, τ )) for the quenched QCD at finite temperature. The measurement of the renormalized EMT in the gradient flow method reduces a statistical uncertainty thanks to its property of the smearing. However, the smearing breaks the sum rule of the spectral function and the over-smeared data in the correlation function may have to be eliminated from the analyzing process of physical observables. In this work, we demonstrate the sparse modeling analysis in the intermediate-representation basis (IR basis), which connects between the Matsubara frequency data and real frequency data. It works well even using very limited data of C (t, τ ) only in the fiducial window of the gradient flow. We utilize the ADMM algorithm which is useful to solve the LASSO problem under some constraints. We show that the obtained spectral function reproduces the input smeared correlation function at finite flow-time. Several systematic and statistical errors and the flow-time dependence are also discussed.

Structure learning with similarity preserving

Leveraging on the underlying low-dimensional structure of data, low-rank and sparse modeling approaches have achieved great success in a wide range of applications. However, in many applications the data can display structures beyond simply being low-rank or sparse. Fully extracting and exploiting hidden structure information in the data is always desirable and favorable. To reveal more underlying effective manifold structure, in this paper, we explicitly model the data relation. Specifically, we propose a structure learning framework that retains the pairwise similarities between the data points. Rather than just trying to reconstruct the original data based on self-expression, we also manage to reconstruct the kernel matrix, which functions as similarity preserving. Consequently, this technique is particularly suitable for the class of learning problems that are sensitive to sample similarity, e.g., clustering and semisupervised classification. To take advantage of representation power of deep neural network, a deep auto-encoder architecture is further designed to implement our model. Extensive experiments on benchmark data sets demonstrate that our proposed framework can consistently and significantly improve performance on both evaluation tasks. We conclude that the quality of structure learning can be enhanced if similarity information is incorporated.

Sparse modeling approach for identifying the dominant factors affecting situation-dependent hourly electricity demand

In recent years, electricity power system structures have changed to accommodate large-scale penetration of renewable energy sources and energy-saving trends. Because of these system changes, the behavior of electricity net-loads, i.e. the difference between total demand and aggregate supply of distributed variable renewable energy sources, has changed drastically. This behavior is also influenced by various other factors such as weather, economic conditions, and consumer lifestyle. Analyzing the factors that affect the dynamic characteristics of the electricity net-load enables stakeholders to construct an electricity business strategy. Conventionally, such demand analysis has been conducted by targeting a limited number of explanatory variables that have been screened according to the prior knowledge of experts. The identification of essential explanatory variables through data-centric analysis, with a focus on variables that co-occur with demand, has long been recognized as important; however, discussion has been limited because it is difficult to describe plausible statistical relationships among the many possible explanatory variables with only a limited number of historical data samples available. This study focuses on the dynamics in hourly electricity demand and approach to identify annually important variables by constructing situation-dependent models. These models are based on the dataset consisting of demand and multiple explanatory variables that co-occur in the target time slices. The powerful concept of sparse modeling is applied to handle the large number of possible explanatory variables used in the situation-dependent modeling process. In particular, this study discusses inconsistency of the selected variables when the statistical models are constructed focusing on different data subsets; when the model is trained based on a dataset focusing on a specific time period, the selected variables may be significantly different from those resulting from a dataset focusing on another time period. The authors propose to derive a limited number of annually dominant variables by enumerating suboptimal models for each situation, and by selecting, as much as possible, essential variables that are commonly and consistently used for all situations. The proposed scheme was applied to a real-world demand dataset and discussed in the context of representation errors and interpretability. The results show that the proposed method is an effective approach for representing the situation-dependent impact of variables on demand.

Valence-selective local atomic structures in inorganic materials by X-ray fluorescence holography

X-ray fluorescence holography (XFH) can be used to conduct atom-resolved structural characterization of materials around a specific element, and has been applied to various functional materials. Recently, a valence-selective function has been found by this technique by employing incident X-ray energies near an absorption edge of a specified element. In this article, the principle and experimental procedure of a valence-selective XFH and subsequent data analysis procedure using the sparse modeling approach of L1 regression are introduced. Then, the excellent XFH results with valence-selective studies are reviewed, such as Y oxide thin film, YbInCu4 valence transition material, and Fe3O4 mixed valence material.

Prediction of appointment no-shows using electronic health records

ABSTRACTAppointment no-shows have a negative impact on patient health and have caused substantial loss in resources and revenue for health care systems. Intervention strategies to reduce no-show rates can be more effective if targeted to the subpopulations of patients with higher risk of not showing to their appointments. We use electronic health records (EHR) from a large medical center to predict no-show patients based on demographic and health care features. We apply sparse Bayesian modeling approaches based on Lasso and automatic relevance determination to predict and identify the most relevant risk factors of no-show patients at a provider level.

Low-rank and sparse subspace modeling of speech for DNN based acoustic modeling

Towards the goal of improving acoustic modeling for automatic speech recognition (ASR), this work investigates the modeling of senone subspaces in deep neural network (DNN) posteriors using low-rank and sparse modeling approaches. While DNN posteriors are typically very high-dimensional, recent studies have shown that the true class information is actually embedded in low-dimensional subspaces. Thus, a matrix of all posteriors belonging to a particular senone class is expected to have a very low rank. In this paper, we exploit Principal Component Analysis and Compressive Sensing based dictionary learning for low-rank and sparse modeling of senone subspaces respectively. Our hypothesis is that the principal components of DNN posterior space (termed as eigen-posteriors in this work) and Compressive Sensing dictionaries can act as suitable models to extract the well-structured low-dimensional latent information and discard the undesirable high-dimensional unstructured noise present in the posteriors. Enhanced DNN posteriors thus obtained are used as soft targets for training better acoustic models to improve ASR. In this context, our approach also enables improving distant speech recognition by mapping far-field acoustic features to low-dimensional senone subspaces learned from near-field features. Experiments are performed on AMI Meeting corpus in both close-talk (IHM) and far-field (SDM) microphone settings where acoustic models trained using enhanced DNN posteriors outperform the conventional hard target based hybrid DNN-HMM systems. An information theoretic analysis is also presented to show how low-rank and sparse enhancement modify the DNN posterior space to better match the assumptions of hidden Markov model (HMM) backend.

The fusion of panchromatic and multispectral remote sensing images via tensor-based sparse modeling and hyper-Laplacian prior

In this paper, we propose a tensor-based non-convex sparse modeling approach for the fusion of panchromatic and multispectral remote sensing images, and this kind of fusion is generally called pansharpening. We first upsample the low spatial-resolution multispectral image by a classical interpolation method to get an initial upsampled multispectral image. Based on the hyper-Laplacian distribution of errors between the upsampled multispectral image and the ground-truth high resolution multispectral image on gradient domain, we formulate a ℓp(0 < p < 1)-norm term to more reasonably describe the relation of these two datasets. In addition, we also model a tensor-based weighted fidelity term for the panchromatic and low resolution multispectral images, aiming to recover more spatial details. Moreover, total variation regularization is also employed to depict the sparsity of the latent high resolution multispectral image on the gradient domain. For the model solving, we design an alternating direction method of multipliers based algorithm to efficiently solve the proposed model. Furthermore, the involved non-convex ℓp subproblem is handled by an efficient generalized shrinkage/thresholding algorithm. Finally, extensive experiments on many datasets collected by different sensors demonstrate the effectiveness of our method when compared with several state-of-the-art image fusion approaches.

Estimation of neuronal dynamics based on sparse modeling

Elucidating neural dynamics is one of the important subjects in neuroscience. To elucidate nonlinear dynamics of single neurons, it is important to extract nonlinear membrane currents from many types of membrane current candidates. In this study, we propose a sparse modeling method for estimating a conductance-based neuron model from observed data, by extracting necessary membrane currents from multiple candidates. We show using simulated data that our proposed sparse modeling approach with different sparsity levels for distinct membrane currents extracts only necessary membrane currents from candidates more accurately, compared with least-squares method and sparse method with uniform sparsity level.

Applications of a L1‐Regularized Linear Regression to X‐Ray Fluorescence Holography Data of Functional Materials

To clarify atom‐resolved structural characterizations of materials, X‐ray fluorescence holography (XFH) technique is employed for drawing three‐dimensional (3D) atomic images around a specific element emitting fluorescent X‐rays. By taking the angle dependences of the fluorescent X‐ray intensity (hologram), 3D images of the surrounding atoms can be, in principle, obtained via simple Fourier transform‐like approaches with no special atomic models. In reality, however, an infinite number of the holograms with different incident X‐ray energies are necessary to reproduce the artifact‐less atomic images at the correct positions. Instead, here an inverse problem is applied using a sparse modeling approach of a L1‐regularized linear regression to solve such statistical problem of a small data size. The excellent results of this approach are presented on complex crystals of some functional materials, such as Mn doped Bi2Te3 topological insulator, Fe65Ni35 Invar alloy, and Fe chalcogenide high‐temperature superconductor.