Relevant Subspace Research Articles

One-Particle Operator Representation over Two-Particle Basis Sets for Relativistic QED Computations.

This work is concerned with two-spin-1/2-fermion relativistic quantum mechanics, and it is about the construction of one-particle projectors using an inherently two-particle, "explicitly correlated" basis representation necessary for good numerical convergence of the interaction energy. It is demonstrated that a faithful representation of the one-particle operators, which appear in intermediate but essential computational steps, can be constructed over a many-particle basis set by accounting for the full Hilbert space beyond the physically relevant antisymmetric subspace. Applications of this development can be foreseen for the computation of quantum-electrodynamics corrections for a correlated relativistic reference state and high-precision relativistic computations of medium-to-high-Z helium-like systems, for which other two-particle projection techniques are unreliable.

Quadrature formulas on combinatorial graphs

The goal of the paper is to establish quadrature formulas on combinatorial graphs. Three types of quadrature formulas are developed. Quadrature formulas of the first type are obtained through interpolation by variational splines. This set of formulas is exact on spaces of variational splines on graphs. Since bandlimited functions can be obtained as limits of variational splines we obtain quadrature formulas which are approximately exact on spaces of bandlimited functions. Accuracy of this type of quadrature formulas is given in terms of geometry of the set of nodes of splines and in terms of smoothness of functions which is measured by means of the combinatorial Laplace operator. Quadrature formulas of the second type are obtained through point-wise sampling for bandlimited functions and based on existence of certain frames in appropriate subspaces of bandlimited functions. The third type quadrature formulas are based on the average sampling over subgraphs. Our quadrature formulas which are based on sampling are exact on a relevant subspaces of bandlimited functions. The results of the paper have potential applications to problems that arise in data mining.

Disentangled Explanations of Neural Network Predictions by Finding Relevant Subspaces.

Explainable AI aims to overcome the black-box nature of complex ML models like neural networks by generating explanations for their predictions. Explanations often take the form of a heatmap identifying input features (e.g. pixels) that are relevant to the model's decision. These explanations, however, entangle the potentially multiple factors that enter into the overall complex decision strategy. We propose to disentangle explanations by extracting at some intermediate layer of a neural network, subspaces that capture the multiple and distinct activation patterns (e.g. visual concepts) that are relevant to the prediction. To automatically extract these subspaces, we propose two new analyses, extending principles found in PCA or ICA to explanations. These novel analyses, which we call principal relevant component analysis (PRCA) and disentangled relevant subspace analysis (DRSA), maximize relevance instead of e.g. variance or kurtosis. This allows for a much stronger focus of the analysis on what the ML model actually uses for predicting, ignoring activations or concepts to which the model is invariant. Our approach is general enough to work alongside common attribution techniques such as Shapley Value, Integrated Gradients, or LRP. Our proposed methods show to be practically useful and compare favorably to the state of the art as demonstrated on benchmarks and three use cases.

On Krylov methods for large-scale CBCT reconstruction.

Krylov subspace methods are a powerful family of iterative solvers for linear systems of equations, which are commonly used for inverse problems due to their intrinsic regularization properties. Moreover, these methods are naturally suited to solve large-scale problems, as they only require matrix-vector products with the system matrix (and its adjoint) to compute approximate solutions, and they display a very fast convergence. Even if this class of methods has been widely researched and studied in the numerical linear algebra community, its use in applied medical physics and applied engineering is still very limited. e.g. in realistic large-scale computed tomography (CT) problems, and more specifically in cone beam CT (CBCT). This work attempts to breach this gap by providing a general framework for the most relevant Krylov subspace methods applied to 3D CT problems, including the most well-known Krylov solvers for non-square systems (CGLS, LSQR, LSMR), possibly in combination with Tikhonov regularization, and methods that incorporate total variation regularization. This is provided within an open source framework: the tomographic iterative GPU-based reconstruction toolbox, with the idea of promoting accessibility and reproducibility of the results for the algorithms presented. Finally, numerical results in synthetic and real-world 3D CT applications (medical CBCT andμ-CT datasets) are provided to showcase and compare the different Krylov subspace methods presented in the paper, as well as their suitability for different kinds of problems.

Exploiting subspace constraints and ab initio variational methods for quantum chemistry

Variational methods offer a highly promising route to exploiting quantum computers for chemistry tasks. Here we employ methods described in a sister paper to the present report, entitled exploring ab initio machine synthesis of quantum circuits, in order to solve problems using adaptively evolving quantum circuits. Consistent with prior authors we find that this approach can outperform human-designed circuits such as the coupled-cluster or hardware-efficient ansätze, and we make comparisons for larger instances up to 14 qubits Moreover we introduce a novel approach to constraining the circuit evolution in the physically relevant subspace, finding that this greatly improves performance and compactness of the circuits. We consider both static and dynamics properties of molecular systems. The emulation environment used is QuESTlink all resources are open source and linked from this paper.

Visual Analytics of Co-Occurrences to Discover Subspaces in Structured Data

We present an approach that shows all relevant subspaces of categorical data condensed in a single picture. We model the categorical values of the attributes as co-occurrences with data partitions generated from structured data using pattern mining. We show that these co-occurrences are a-priori , allowing us to greatly reduce the search space, effectively generating the condensed picture where conventional approaches filter out several subspaces as these are deemed insignificant. The task of identifying interesting subspaces is common but difficult due to exponential search spaces and the curse of dimensionality. One application of such a task might be identifying a cohort of patients defined by attributes such as gender, age, and diabetes type that share a common patient history, which is modeled as event sequences. Filtering the data by these attributes is common but cumbersome and often does not allow a comparison of subspaces. We contribute a powerful multi-dimensional pattern exploration approach (MDPE-approach) agnostic to the structured data type that models multiple attributes and their characteristics as co-occurrences, allowing the user to identify and compare thousands of subspaces of interest in a single picture. In our MDPE-approach, we introduce two methods to dramatically reduce the search space, outputting only the boundaries of the search space in the form of two tables. We implement the MDPE-approach in an interactive visual interface (MDPE-vis) that provides a scalable, pixel-based visualization design allowing the identification, comparison, and sense-making of subspaces in structured data. Our case studies using a gold-standard dataset and external domain experts confirm our approach’s and implementation’s applicability. A third use case sheds light on the scalability of our approach and a user study with 15 participants underlines its usefulness and power.

Multianalyte Serum Biomarker Panel for Early Detection of Pancreatic Adenocarcinoma.

We determined whether a large, multianalyte panel of circulating biomarkers can improve detection of early-stage pancreatic ductal adenocarcinoma (PDAC). We defined a biologically relevant subspace of blood analytes on the basis of previous identification in premalignant lesions or early-stage PDAC and evaluated each in pilot studies. The 31 analytes that met minimum diagnostic accuracy were measured in serum of 837 subjects (461 healthy, 194 benign pancreatic disease, and 182 early-stage PDAC). We used machine learning to develop classification algorithms using the relationship between subjects on the basis of their changes across the predictors. Model performance was subsequently evaluated in an independent validation data set from 186 additional subjects. A classification model was trained on 669 subjects (358 healthy, 159 benign, and 152 early-stage PDAC). Model evaluation on a hold-out test set of 168 subjects (103 healthy, 35 benign, and 30 early-stage PDAC) yielded an area under the receiver operating characteristic curve (AUC) of 0.920 for classification of PDAC from non-PDAC (benign and healthy controls) and an AUC of 0.944 for PDAC versus healthy controls. The algorithm was then validated in 146 subsequent cases presenting with pancreatic disease (73 benign pancreatic disease and 73 early- and late-stage PDAC cases) and 40 healthy control subjects. The validation set yielded an AUC of 0.919 for classification of PDAC from non-PDAC and an AUC of 0.925 for PDAC versus healthy controls. Individually weak serum biomarkers can be combined into a strong classification algorithm to develop a blood test to identify patients who may benefit from further testing.

Structured fault information-aided canonical variate analysis model for dynamic process monitoring

Process monitoring is one of the most crucial fundamental components in industrial processes. Traditional multivariate statistical analysis modeling only relies on data collected in normal condition, reflecting the lack of sensitivity to faults. This paper proposed a fault information-aided canonical variate analysis (FICVA) and a structured FICVA (SFICVA) monitoring strategy to improve fault detection rate. In FICVA, canonical variate analysis (CVA) is firstly applied to extract state and residual subspace based on normal data. Then, according to the data collected from one fault, the above subspaces are further decomposed into fault relevant and irrelevant state subspaces, and fault relevant and irrelevant residual subspaces for guaranteeing fault information can be concentrated. The correlation between normal data in various subspaces and the ability to concentrate abnormal information in fault data from fault relevant subspaces are theoretically analyzed. Then, to combine diverse fault information from different scenarios, SFICVA monitoring strategy is performed through Bayesian inference to ensemble a series of FICVA sub-models constructed by different faults to obtain more effective performance. Through a numerical example, experimental research on Tennessee Eastman process (TEP) and actual blast furnace ironmaking process (BFIP), rationality of FICVA method and effectiveness of SFICVA monitoring strategy are fully verified.

Multi-Class Support Vector Machines with Incremental Learning for Text Classification

Class-incremental learning (CIL) is a revolutionary framework we develop in this study to address multi-class problems with support vector machines (SVM). Text classifiers built with support for support vector machines (SVMs) can be kept up-to-date with the help of CIL’s two incremental processes. Reusing previously learned classifier models, the CIL only needs to train a single binary sub-classifier and an extra step for feature assortmentonce a new class is introduced. The projections of the vectors onto the relevant subspaces are analyzed using the present classifier. Any text classification method based on binary classification can use CIL as a universal framework for implementation. We found that the CIL-based SVM not only outperformed well-known batch SVM learning strategies like 1-against-rest, 1-against-1, and divide-by-2, but also required much less time to train.

Fast Relevant Feature Selection Based on Markov Boundary Learning from Data Stream for Classification Tasks

Learning a classification model from a dataset with the presence of irrelevant variables makes such a task more difficult and less accurate. To address this issue, feature selection has been proposed. However, existing feature selection struggles to select accurate and relevant sub-features or subspaces. It has been demonstrated in the literature that learning the Markov boundary (MB) of features helps to extract appropriate subsets of relevant features from the full data space. However, due to the fact that almost all existing methods learn MBs with static data, a state-of-the-art algorithm for learning MBs with streaming data, called SDMB (streaming data-based MB), has recently been proposed. SDMB has successfully improved online feature selection based on learning Markov boundary (MB) from data stream for classification tasks. Nevertheless, we found that SDMB has computational redundancy and there is room for improvement in terms of runtime. Therefore, we propose an improvement to SDMB. The main drawback of SDMB is that it entirely recomputes the conditional independence/dependence tests (i.e., the dependency scores) between a target variable and other variables in each iteration of learning MBs from the full data space. This paper proposes an incremental SDMB, called FastSDMB, to address the above bottleneck by reducing the dependency score computation time of SDMB. FastSDMB avoids the computational redundancy in SDMB by using a novel incremental computing collective dependency scoring process. This incremental process retains and reuses, in current iterations, the calculated conditional independence test score of variables that have not changed from previous iterations. To demonstrate FastSDMB's effectiveness and capability of handling fast data arrival scenarios, comparisons are made first over the state-of-the-art SDMB, and then we apply FastSDMB to various classification models in order to showcase the resulting performance improvement when implemented over a wide variety of synthetic and real-world datasets. The experimental results show that FastSDMB can run up to 4.8 × faster on average than SDMB and it maintains the same accuracy when learning Markov boundary. Thereby proving its efficiency in fast relevant feature selection based on Markov boundary learning from data stream.

Protein Structure Validation Derives a Smart Conformational Search in a Physically Relevant Configurational Subspace.

Since proteins perform biological functions through their dynamic properties, molecular dynamics (MD) simulation is a sophisticated strategy for investigating their functions. Analyses of trajectories provide statistical information about a specific protein as a free-energy landscape (FEL). However, the timescale of normal MD is shorter than that of biological functions, resulting in statistically insufficient conformational sampling, finally leading to unreliable FEL calculation. To search for a broad configurational subspace, an external bias is imposed on a target protein as biased sampling. However, its regulation is challenging because the optimal strength of the perturbation is unknown. Furthermore, a physically irrelevant configurational subspace was searched when imposing an inappropriate external bias. To address this issue, we newly proposed an external biased regulation scheme known as the G-factor external bias limiter (GERBIL). In GERBIL, protein configurations generated by external bias are structurally validated by an indicator (G-factor), enabling the search for a physically relevant subspace. In addition to biased sampling, nonbiased sampling might search for a physically irrelevant configurational subspace because repeating multiple MD simulations from several initial structures tends to search for an overly broad configurational subspace. For this issue, the structural qualities of configurations generated by nonbiased sampling have not been investigated. Therefore, we confirmed whether the G-factor screened the collapsed (low-quality) configurations generated by nonbiased sampling. To address this issue, the outlier flooding method (OFLOOD) was adopted in GERBIL as a nonbiased sampling method, which is referred to as OFLOOD-GERBIL. OFLOOD rapidly expands a configurational subspace by resampling the rarely occurring states of a given protein and tends to search an overly broad subspace. Thus, we considered that GERBIL might improve the excessive conformational search of OFLOOD for a physically irrelevant configurational subspace. As a demonstration, OFLOOD and OFLOOD-GERBIL were applied to a globular protein (T4 lysozyme) and their conformational search qualities were assessed. Based on our assessment, normal OFLOOD without the outlier validation frequently sampled low-quality configurations, whereas OFLOOD-GERBIL with the outlier validation intensively sampled high-quality configurations. In conclusion, OFLOOD-GERBIL derives a smart conformational search in a physically relevant configurational subspace, indicating that protein structure validation works in both nonbiased and biased sampling methods.

Speech listening entails neural encoding of invisible articulatory features

Speech processing entails a complex interplay between bottom-up and top-down computations. The former is reflected in the neural entrainment to the quasi-rhythmic properties of speech acoustics while the latter is supposed to guide the selection of the most relevant input subspace. Top-down signals are believed to originate mainly from motor regions, yet similar activities have been shown to tune attentional cycles also for simpler, non-speech stimuli. Here we examined whether, during speech listening, the brain reconstructs articulatory patterns associated to speech production. We measured electroencephalographic (EEG) data while participants listened to sentences during the production of which articulatory kinematics of lips, jaws and tongue were also recorded (via Electro-Magnetic Articulography, EMA). We captured the patterns of articulatory coordination through Principal Component Analysis (PCA) and used Partial Information Decomposition (PID) to identify whether the speech envelope and each of the kinematic components provided unique, synergistic and/or redundant information regarding the EEG signals. Interestingly, tongue movements contain both unique as well as synergistic information with the envelope that are encoded in the listener's brain activity. This demonstrates that during speech listening the brain retrieves highly specific and unique motor information that is never accessible through vision, thus leveraging audio-motor maps that arise most likely from the acquisition of speech production during development.

Coherent control of the vibrational dynamics of aligned heteronuclear diatomic molecules

We present an analytical pulse design protocol for controlling the vibrational dynamics of polar diatomic molecules within a given electronic state. Altering the potential energy function via the position-dependent electric permanent dipole moment, the vibrational state population dynamics is directly controlled using appropriately shaped laser pulses in the midinfrared regime. The optimal pulse shapes---that are expected to drive the molecule along user-defined quantum pathways---are obtained by reverse engineering, that is, solving the Schr\"odinger equation of the nuclei inversely in a relevant subspace. The proposed control scheme is validated by accurately solving the full time-dependent Schr\"odinger equation of the ${\mathrm{HeH}}^{+}$ molecular ion with two completely different methods: (1) propagating the complex population amplitudes of many field-free eigenstates or (2) propagating directly the nuclear wave packet on a grid. We find that besides smooth transitions, arbitrary Rabi oscillations as well as vibrational ladder climbing can be efficiently controlled with the present scheme. As a result, the molecule is successively excited beyond the potential barrier, leading to enhanced dissociation in the ground electronic state. Rotational effects and possible extensions of the presented control are also briefly discussed.

Nonconvex multi-view subspace clustering via simultaneously learning the representation tensor and affinity matrix* *This research was supported by the National Natural Science Foundations of China (12071159, U1811464) and the NSF-DMS 1854638 of the United States.

Multi-view subspace clustering, which aims to partition a dataset into its relevant subspaces based on their multi-view features, has been widely applied to identify various characteristics of datasets. The typical model of multi-view subspace clustering in literature often makes use of the nuclear norm to seek the underlying low-rank representation. However, due to the sum property of the singular values defined by tensor nuclear norm, the existing multi-view subspace clustering does not well handle the noise and the illumination variations embedded in multi-view data. To address and improve the robustness and clustering performance, we propose a new nonconvex multi-view subspace clustering model via tensor minimax concave penalty (MCP) approximation associated with rank minimization (NMSC-MCP), which can simultaneously construct the low-rank representation tensor and affinity matrix in a unified framework. Specifically, the nonconvex MCP approximation rank function is adopted to as a tighter tensor rank approximation to discriminate the dimension of features so that better accuracy can be achieved. In addition, we also address the local structure by including both hyper-Laplacian regularization and auto-weighting scheme into the objective function to promote the clustering performance. A corresponding iterative algorithm is then developed to solve the proposed model and the constructed iterative sequence generated by the proposed algorithm is shown to converge to the desirable KKT critical point. Extensive experiments on benchmark datasets have demonstrate the highly desirable effectiveness of our proposed method.

An approach for unsupervised contextual anomaly detection and characterization

Outlier detection has been widely explored and applied to different real-world problems. However, outlier characterization that consists in finding and understanding the outlying aspects of the anomalous observations is still challenging. In this paper, we present a new approach to simultaneously detect subspace outliers and characterize them. We introduce the Dimension-wise Local Outlier Factor (DLOF) function to quantify the degree of outlierness of the data points in each feature dimension. The obtained DLOFs are used in an outlier ensemble so as to detect and rank the anomalous points. Subsequently, the same DLOFs are analyzed in order to characterize the detected outliers with their relevant subspace and their same-type anomalies. Experiments on various datasets show the efficacy of our method. Indeed, we demonstrate through an experimental evaluation that the proposed approach is competitive compared to the existing solutions in terms of both detection and characterization accuracy.

The sloppy relationship between neural circuit structure and function.

Investigating and describing the relationships between the structure of a circuit and its function has a long tradition in neuroscience. Since neural circuits acquire their structure through sophisticated developmental programmes, and memories and experiences are maintained through synaptic modification, it is to be expected that structure is closely linked to function. Recent findings challenge this hypothesis from three different angles: function does not strongly constrain circuit parameters, many parameters in neural circuits are irrelevant and contribute little to function, and circuit parameters are unstable and subject to constant random drift. At the same time, however, recent work also showed that dynamics in neural circuit activity that is related to function are robust over time and across individuals. Here this apparent contradiction is addressed by considering the properties of neural manifolds that restrict circuit activity to functionally relevant subspaces, and it will be suggested that degenerate, anisotropic and unstable parameter spaces are closely related to the structure and implementation of functionally relevant neural manifolds.

Subspace-based outlier detection using linear programming and heuristic techniques

A useful strategy to perform outlier detection (OD) in highdimensional data, especially in the presence of multiple classes of outliers, is to decompose the outlier detection problem into a set of relevant subspace selection (RSS) problems. In this approach, one relevant subspace is selected locally for each data point, and then the outlierness of that data point within its relevant subspaces is evaluated. The k-nearest neighbor algorithm has been used in most local-based subspace selection methods for OD application in high dimensional data. However, such techniques suffer from the curse of dimensionality, exhibit high computational complexity, and also, it may not be suitable for arbitrary distributions of the data. Additionally, the subspace nearest neighbor search (SNNS) problem in high dimensional data is rarely discussed in these methods. In this paper, we formulate the local selection of relevant subspace method for OD problem that employs a linear programming (LP) model with the objective function of local sparsity maximization. Our formulation for local selection of relevant subspace as an LP also considers the subspace nearest neighbor search problem to simultaneously solve these two subproblems (local selection of relevant subspace and subspace nearest neighbor search) that are tightly coupled. In addition, a novel heuristic algorithm, with linear complexity with respect to the number of dimensions, is developed to solve the LP model through an iterative procedure that alternates between two algorithms of subspace nearest neighbor search and RSS. Experimental results on both synthetic and real datasets show the feasibility of our formulation, the effectiveness of the heuristic algorithm, and also show the superiority of our proposed subspace outlier detection algorithm based on linear programming and heuristic techniques (SODLPH) to other published studies in terms of detection accuracy. Assessing by the average of the AUC on seven real-world UCI datasets, the performance of our proposed method has improved at least 8.72% in comparison with other related published works.

Estimating the Euclidean quantum propagator with deep generative modeling of Feynman paths

Feynman path integrals provide an elegant, classically inspired representation for the quantum propagator and the quantum dynamics, through summing over a huge manifold of all possible paths. From computational and simulational perspectives, the ergodic tracking of the whole path manifold is a hard problem. Machine learning can help, in an efficient manner, to identify the relevant subspace and the intrinsic structure residing at a small fraction of the vast path manifold. In this work, we propose the Feynman path generator for quantum mechanical systems, which efficiently generates Feynman paths with fixed endpoints, from a (low-dimensional) latent space and by targeting a desired density of paths in the Euclidean space-time. With such path generators, the Euclidean propagator as well as the ground-state wave function can be estimated efficiently for a generic potential energy. Our work provides an alternative approach for calculating the quantum propagator and the ground-state wave function, paves the way toward generative modeling of quantum mechanical Feynman paths, and offers a different perspective to understand the quantum-classical correspondence through deep learning.

An adaptive eigenfunction basis strategy to reduce design dimension in topology optimization

AbstractThe concept of adaptive eigenspace basis (AEB) has recently proved effective for solving medium imaging problems. In this article, we present an AEB strategy for design parameterization in topology optimization (TO) problems. We seek the density design field as a linear combination of eigenfunctions, computed for an elliptic operator defined over the structural domain, and solve for the associated eigenfunction coefficients. Restriction to this truncated eigenspace drastically reduces the design dimension and imposes implicit regularization upon the solution, removing the need for auxiliary filtering operations and design‐variable bound constraints. We furthermore develop the basis adaptation scheme inherent in the AEB, which iteratively recomputes the eigenfunction basis to conform to the evolving density field, enabling further dimension reduction and acceleration of the optimization process. The known aptitude of the adapted eigenfunctions to approximate piecewise constant fields is especially useful for TO as relevant design subspaces can be given low‐dimensional representations. We propose criteria for the selection of the basis dimension and demonstrate the use of basis function selection as means for length scale control. We compare performance of the AEB against conventional TO implementations in problems for static linear‐elasticity, showing comparable structural solutions, computational cost benefits, and consistent design dimension reduction.

Margin-Based Feature Selection in Incomplete Data

This study considers the problem of feature selection in incomplete data. The intuitive approach is to first impute the missing values, and then apply a standard feature selection method to select relevant features. In this study, we show how to perform feature selection directly, without imputing missing values. We define the objective function of the uncertainty margin-based feature selection method to maximize each instance’s uncertainty margin in its own relevant subspace. In optimization, we take into account the uncertainty of each instance due to the missing values. The experimental results on synthetic and 6 benchmark data sets with few missing values (less than 25%) provide evidence that our method can select the same accurate features as the alternative methods which apply an imputation method first. However, when there is a large fraction of missing values (more than 25%) in data, our feature selection method outperforms the alternatives, which impute missing values first.