Subspace Regularization Research Articles

Multiple Heterogeneous Networks Representation With Latent Space for Synthetic Lethality Prediction.

Computational synthetic lethality (SL) method has become a promising strategy to identify SL gene pairs for targeted cancer therapy and cancer medicine development. Feature representation for integrating various biological networks is crutial to improve the identification performance. However, previous feature representation, such as matrix factorization and graph neural network, projects gene features onto latent variables by keeping a specific geometric metric. There is a lack of models of gene representational latent space with considerating multiple dimentionalities correlation and preserving latent geometric structures in both sample and feature spaces. Therefore, we propose a novel method to model gene Latent Space using matrix Tri-Factorization (LSTF) to obtain gene representation with embedding variables resulting from the potential interpretation of synthetic lethality. Meanwhile, manifold subspace regularization is applied to the tri-factorization to capture the geometrical manifold structure in the latent space with gene PPI functional and GO semantic embeddings. Then, SL gene pairs are identified by the reconstruction of the associations with gene representations in the latent space. The experimental results illustrate that LSTF is superior to other state-of-the-art methods. Case study demonstrate the effectiveness of the predicted SL associations.

Domain-Specific Processing Stage for Estimating Single-Trail Evoked Potential Improves CNN Performance in Detecting Error Potential.

We present a novel architecture designed to enhance the detection of Error Potential (ErrP) signals during ErrP stimulation tasks. In the context of predicting ErrP presence, conventional Convolutional Neural Networks (CNNs) typically accept a raw EEG signal as input, encompassing both the information associated with the evoked potential and the background activity, which can potentially diminish predictive accuracy. Our approach involves advanced Single-Trial (ST) ErrP enhancement techniques for processing raw EEG signals in the initial stage, followed by CNNs for discerning between ErrP and NonErrP segments in the second stage. We tested different combinations of methods and CNNs. As far as ST ErrP estimation is concerned, we examined various methods encompassing subspace regularization techniques, Continuous Wavelet Transform, and ARX models. For the classification stage, we evaluated the performance of EEGNet, CNN, and a Siamese Neural Network. A comparative analysis against the method of directly applying CNNs to raw EEG signals revealed the advantages of our architecture. Leveraging subspace regularization yielded the best improvement in classification metrics, at up to 14% in balanced accuracy and 13.4% in F1-score.

Spatiotemporal traffic data imputation by synergizing low tensor ring rank and nonlocal subspace regularization

AbstractSpatiotemporal traffic data usually suffers from missing entries in the data acquisition and transmission process. Existing imputation methods only consider the global/local structure of spatiotemporal traffic data, resulting in insufficient estimation performance. Fortunately, it is found that traffic data admits the nonlocal self‐similarity (NSS) prior. This paper incorporates the global and nonlocal low‐rank priors of traffic data and proposes a tensor completion model for spatiotemporal traffic data imputation. To be specific, the proposed method uses tensor ring (TR) decomposition with an enhanced representation capability to characterize the global low‐TR‐rank prior of traffic data, e.g. the correlation of sensor and time modes of the tensor (i.e. traffic data). An implicit plug‐and‐play (PnP)‐based regularization is further utilized to exploit the NSS prior, which depicts the nonlocal similar traffic data patterns. Furthermore, the proximal alternating minimization algorithm under the PnP framework is derived to solve this model. The experiment results on various datasets and missing scenarios show the superiority of the proposed model.

DIAS: A Data-Informed Active Subspace Regularization Framework for Inverse Problems

This paper presents a regularization framework that aims to improve the fidelity of Tikhonov inverse solutions. At the heart of the framework is the data-informed regularization idea that only data-uninformed parameters need to be regularized, while the data-informed parameters, on which data and forward model are integrated, should remain untouched. We propose to employ the active subspace method to determine the data-informativeness of a parameter. The resulting framework is thus called a data-informed (DI) active subspace (DIAS) regularization. Four proposed DIAS variants are rigorously analyzed, shown to be robust with the regularization parameter and capable of avoiding polluting solution features informed by the data. They are thus well suited for problems with small or reasonably small noise corruptions in the data. Furthermore, the DIAS approaches can effectively reuse any Tikhonov regularization codes/libraries. Though they are readily applicable for nonlinear inverse problems, we focus on linear problems in this paper in order to gain insights into the framework. Various numerical results for linear inverse problems are presented to verify theoretical findings and to demonstrate advantages of the DIAS framework over the Tikhonov, truncated SVD, and the TSVD-based DI approaches.

Wavelet-Based Subspace Regularization for Solving Highly Nonlinear Inverse Scattering Problems with Contraction Integral Equation

A wavelet transform twofold subspace-based optimization method (WT-TSOM) is proposed to solve the highly nonlinear inverse scattering problems with contraction integral equation for inversion (CIE-I). While the CIE-I is able to suppress the multiple scattering effects within inversion (without compromising the accuracy of the physics), proper regularization is needed. In this paper, we investigate a new type subspace regularization technique based on wavelet expansions for the induced currents. We found that the bior3.5 wavelet is a good choice to stabilize the inversions with the CIE-I model and in the meanwhile it also can rectify the contrast profile. Numerical tests against both synthetic and experimental data show that WT-TSOM is a promising regularization technique for inversion with CIE-I.

Learning shared subspace regularization with linear discriminant analysis for multi-label action recognition

Human action recognition under complex environment is a challenging work, while in deep learning and in these specific difficulty recognition tasks, the multi-label linear discriminant analysis (MLDA) is already utilized. As is known to all, MLDA is used for dimensionality feature reduction. Nevertheless, MLDA contains the eigendecomposition of dense matrices, which will cost a huge money on a high dimension computing. In this study, we demonstrate that the MLDA formula is able to equivalent to the least squares problem, which greatly reduces the computation and size of high-dimensional datasets. In addition, it is found that introducing attractive regularization technique into the classical least squares strategy can improve the robustness. The established equivalence relationship is proved by laboratory results on three action datasets. In addition, through comparing with some typical and recent algorithms, the superiority of our constructed model is also demonstrated.

A data-driven regularization approach for template matching in spike sorting with high-density neural probes.

Spike sorting is the process of assigning neural spikes in an extracellular brain recording to their putative neurons. Optimal pre-whitened template matching filters that are used in spike sorting typically suffer from ill-conditioning. In this paper, we investigate the origin of this ill-conditioning and the way in which it influences the resulting filters. Two data-driven subspace regularization approaches are proposed, and those are shown to outperform a regularization approach used in recent literature. The comparison of the methods is based on ground truth data that are recorded in-vivo.

Regularized constraint subspace based method for image set classification

Subspace methods are popular for image set classification due to the excellent representation ability of subspaces. Generalized difference subspace and orthogonal subspace are two currently effective projection strategies for extracting discriminative subspaces. However, both of these methods discard part of the common subspace to form the constraint subspace, which may cause a loss of discriminative information. In this work, we combine the difference subspace and orthogonal subspace to form a full rank constraint subspace. Moreover, we generalize this approach to a common framework using eigenspectrum regularization models (ERMs). The full rank constraint subspace that is regularized by different ERMs is called the regularized constraint subspace (RCS). Furthermore, we propose a new ERM using the concept of difference subspace, namely, the difference subspace regularization model (DSRM). The DSRM and two other current ERMs are incorporated in our RCS-based framework. The results from extensive experiments have demonstrated the effectiveness of our proposed approaches.

Efficient inversion and uncertainty quantification of a tephra fallout model

AbstractAn efficient and effective inversion and uncertainty quantification approach is proposed for estimating eruption parameters given a data set collected from a tephra deposit. The approach is model independent and here is applied using Tephra2, a code that simulates advective and dispersive tephra transport and deposition. The Levenburg‐Marquardt algorithm is combined with formal Tikhonov and subspace regularization to invert eruption parameters; a linear equation for conditional uncertainty propagation is used to estimate posterior parameter uncertainty. Both the inversion and uncertainty analysis support simultaneous analysis of the full eruption and wind field parameterization. The combined inversion/uncertainty quantification approach is applied to the 1992 eruption of Cerro Negro and the 2011 Kirishima‐Shinmoedake eruption. While eruption mass uncertainty is reduced by inversion against tephra isomass data, considerable uncertainty remains for many eruption and wind field parameters, such as plume height. Supplementing the inversion data set with tephra granulometry data is shown to further reduce the uncertainty of most eruption and wind field parameters. The eruption mass of the 2011 Kirishima‐Shinmoedake eruption is 0.82 × 1010 kg to 2.6 × 1010 kg, with 95% confidence; total eruption mass for the 1992 Cerro Negro eruption is 4.2 × 1010 kg to 7.3 × 1010 kg, with 95% confidence. These results indicate that eruption classification and characterization of eruption parameters can be significantly improved through this uncertainty quantification approach.

A new sequential procedure for hydraulic tomographic inversion

We present a novel hydraulic tomography procedure, which is based on a travel-time based inversion and application of pilot points. The travel-time based inversion allows for the reconstruction of a diffusivity distribution, which displays the dominant structural elements of a heterogeneous aquifer. This information is used to guide pilot point based inversion of hydraulic conductivity distribution. We utilize the diffusivity distribution to optimize the spatial positions of the pilot points and the relationships among them. For the implementation of the relationships, graph theory is applied. The associated high-computational effort was encountered with subspace regularization and parallel computing. The developed inverse methodology was successfully applied to a three-dimensional sedimentary aquifer analogue. The validation shows that the sequential procedure strongly improves the misfit between predicted and true pressure response in comparison to the results, only when travel-time inversion is employed.

Linking local and global optical flow computation by subspace regularization

Optical flow computation methods are traditionally classified to two categories: local and global. Several previous works have investigated the combination of them by exploiting their complementary effects. Unlike previous works, this paper is motivated by the common weakness of the two schemes and proposes linking local and global computation by subspace regularization. We will show that the subspace regularization can effectively reduce the error caused by ill-conditioned local computation systems, while inhibiting error diffusion suffered by the global regularization. Experimental results on benchmark sequences validate the enhanced performance of the proposed combination scheme.

Robust classification of face and head gestures in video

Automatic analysis of head gestures and facial expressions is a challenging research area and it has significant applications in human-computer interfaces. We develop a face and head gesture detector in video streams. The detector is based on face landmark paradigm in that appearance and configuration information of landmarks are used. First we detect and track accurately facial landmarks using adaptive templates, Kalman predictor and subspace regularization. Then the trajectories (time series) of facial landmark positions during the course of the head gesture or facial expression are converted in various discriminative features. Features can be landmark coordinate time series, facial geometric features or patches on expressive regions of the face. We use comparatively, two feature sequence classifiers, that is, Hidden Markov Models (HMM) and Hidden Conditional Random Fields (HCRF), and various feature subspace classifiers, that is, ICA (Independent Component Analysis) and NMF (Non-negative Matrix Factorization) on the spatiotemporal data. We achieve 87.3% correct gesture classification on a seven-gesture test database, and the performance reaches 98.2% correct detection under a fusion scheme. Promising and competitive results are also achieved on classification of naturally occurring gesture clips of LIlir TwoTalk Corpus.

Enhancement of the Krylov Subspace Regularization for Microwave Biomedical Imaging

Although Krylov subspace methods provide fast regularization techniques for the microwave imaging problem, they cannot preserve the edges of the object being imaged and may result in an oscillatory reconstruction. To suppress these spurious oscillations and to provide an edge-preserving regularization, we use a multiplicative regularizer which improves the reconstruction results significantly while adding little computational complexity to the inversion algorithm. We show the inversion results for a real human forearm assuming the 2-D transverse magnetic illumination and a cylindrical object assuming the 2-D transverse electric illumination.

Calibration‐constrained Monte Carlo analysis of highly parameterized models using subspace techniques

We describe a subspace Monte Carlo (SSMC) technique that reduces the burden of calibration‐constrained Monte Carlo when undertaken with highly parameterized models. When Monte Carlo methods are used to evaluate the uncertainty in model outputs, ensuring that parameter realizations reproduce the calibration data requires many model runs to condition each realization. In the new SSMC approach, the model is first calibrated using a subspace regularization method, ideally the hybrid Tikhonov‐TSVD “superparameter” approach described by Tonkin and Doherty (2005). Sensitivities calculated with the calibrated model are used to define the calibration null‐space, which is spanned by parameter combinations that have no effect on simulated equivalents to available observations. Next, a stochastic parameter generator is used to produce parameter realizations, and for each a difference is formed between the stochastic parameters and the calibrated parameters. This difference is projected onto the calibration null‐space and added to the calibrated parameters. If the model is no longer calibrated, parameter combinations that span the calibration solution space are reestimated while retaining the null‐space projected parameter differences as additive values. The recalibration can often be undertaken using existing sensitivities, so that conditioning requires only a small number of model runs. Using synthetic and real‐world model applications we demonstrate that the SSMC approach is general (it is not limited to any particular model or any particular parameterization scheme) and that it can rapidly produce a large number of conditioned parameter sets.

Tikhonov regularized solutions for improvement of signal-to-noise ratio in case of auditory-evoked potentials

In the present study, standard Tikhonov regularization (STR) Technique and the subspace regularization (SR) method have been applied to remove the additive EEG noise on average auditory-evoked potential (EP) signals. In methodological manner, the difference between these methods is the formation of regularization matrices which are used to solve the weighted problem of EP estimation. Those methods are compared to ensemble averaging (EA) with respect to signal-to-noise-ratio (SNR) improvement in experimental studies, simulations and pseudo-simulations. The results of tests no superiority of the SR in comparison to STR has been observed. In addition, the STR is found to be less computational complex. Moreover, results support the theoretical fact that the STR was introduced to be optimum for smooth solutions whereas the SR allows sharp variations in solutions. Thus, the STR is found to be more useful in removing the noise with the average signal remaining.

Relationship of P300 single-trial responses with reaction time and preceding stimulus sequence

Variation of single-trial P300 responses was studied both in relation to reaction times and to the preceding stimulus sequence in an auditory oddball paradigm. Single-trial responses were estimated with the Subspace regularization method that is based on Bayesian estimation and linear modeling. The results of the single-trial method were compared to those of averaging. Both methods showed that the latency of the P300 was shorter and its amplitude larger for faster than slower reaction times. The P300 latency was shorter for target tones that were preceded by a large number of standard tones compared to those preceded by a small number of standard tones. The P300 amplitude was statistically significantly affected by the stimulus sequence only when analyzed with conventional averaging. In-depth analysis of standard deviations showed that the variability of the P300 single-trial latencies could explain the differences between the two methods. Specifically, the regression analysis showed that the latency correlated negatively with the number of preceding standard tones and positively with the reaction time, whereas the P300 amplitude correlated positively with the number of the preceding standard stimuli and negatively with the reaction time. The analysis of the single-trial responses gives information about the behavior of the P300 component that is lost with conventional averaging. The method used in this study is independent of subjective decision making and can be used to model changes in the dynamical behavior of the P300 component objectively.

Geometrical interpretation of fMRI-guided MEG/EEG inverse estimates

Magneto- and electroencephalography (MEG/EEG) and functional magnetic resonance imaging (fMRI) provide complementary information about the functional organization of the human brain. An important advantage of MEG/EEG is the millisecond time resolution in detecting electrical activity in the cerebral cortex. The interpretation of MEG/EEG signals, however, is limited by the difficulty of determining the spatial distribution of the neural activity. Functional MRI can help in the MEG/EEG source analysis by suggesting likely locations of activity. We present a geometric interpretation of fMRI-guided inverse solutions in which the MEG/EEG source estimate minimizes a distance to a subspace defined by the fMRI data. In this subspace regularization (SSR) approach, the fMRI bias does not assume preferred amplitudes for MEG/EEG sources, only locations. Characteristic dependence of the source estimates on the regularization parameters is illustrated with simulations. When the fMRI locations match the true MEG/EEG source locations, they serve to bias the underdetermined MEG/EEG inverse solution toward the fMRI loci. Importantly, when the fMRI loci do not match the true MEG/EEG loci, the solution is insensitive to those fMRI loci.

An algorithm using projection onto subspace of prior distributions for long-wavelength sound wave CT.

The stationary long-wavelength sound wave computed tomography is a nonlinear inverse problem that requires the use of prior information of the object. However, the prior assumptions that are usually used in similar inverse problems are more or less inappropriate. In this paper, a new reconstruction algorithm using the prior information is proposed and compared with subspace regularization method and Marquardt reconstruction algorithms. The simulation shows that the proposed algorithm can give a better reconstructed result whether the actual distribution is compatible or incompatible with the prior distributions.

Proximal Point Approach and Approximation of Variational Inequalities

A general approach to analyze convergence of the proximal-like methods for variational inequalities with set-valued maximal monotone operators is developed. It is oriented to methods coupling successive approximation of the variational inequality with the proximal point algorithm as well as to related methods using regularization on a subspace and weak regularization. This approach also covers so-called multistep regularization methods, in which the number of proximal iterations in the approximated problems is controlled by a criterion characterizing these iterations as to be effective. The conditions on convergence require control of the exactness of the approximation only in a certain region of the original space. Conditions ensuring linear convergence of the methods are established.

Subspace regularization method for the single-trial estimation of evoked potentials.

A method for the single-trial estimation of the evoked potentials is proposed. The method is based on the so-called subspace regularization approach in which the second-order statistics of the set of the measurements is used to form a prior information model for the evoked potentials. The method is closely related to the Bayesian estimation. The performance of the proposed method is evaluated using realistic simulations. As a specific application the method is applied to the estimation of the target responses in the P300 test.