Stochastic Proximity Embedding Research Articles

Intrinsic Dimension Estimation-Based Feature Selection and Multinomial Logistic Regression for Classification of Bearing Faults Using Compressively Sampled Vibration Signals.

As failures of rolling bearings lead to major failures in rotating machines, recent vibration-based rolling bearing fault diagnosis techniques are focused on obtaining useful fault features from the huge collection of raw data. However, too many features reduce the classification accuracy and increase the computation time. This paper proposes an effective feature selection technique based on intrinsic dimension estimation of compressively sampled vibration signals. First, compressive sampling (CS) is used to get compressed measurements from the collected raw vibration signals. Then, a global dimension estimator, the geodesic minimal spanning tree (GMST), is employed to compute the minimal number of features needed to represent efficiently the compressively sampled signals. Finally, a feature selection process, combining the stochastic proximity embedding (SPE) and the neighbourhood component analysis (NCA), is used to select fewer features for bearing fault diagnosis. With regression analysis-based predictive modelling technique and the multinomial logistic regression (MLR) classifier, the selected features are assessed in two case studies of rolling bearings vibration signals under different working loads. The experimental results demonstrate that the proposed method can successfully select fewer features, with which the MLR-based trained model achieves high classification accuracy and significantly reduced computation times compared to published research.

Detection of microsleep states from the EEG: a comparison of feature reduction methods.

Microsleeps are brief lapses in consciousness with complete suspension of performance. They are the cause of fatal accidents in many transport sectors requiring sustained attention, especially driving. A microsleep-warning device, using wireless EEG electrodes, could be used to rouse a user from an imminent microsleep. High-dimensional datasets, especially in EEG-based classification, present challenges as there are often a large number of potentially useful features for detecting the phenomenon of interest. Thus, it is often important to reduce the dimension of the original data prior to training the classifier. In this study, linear dimensionality reduction methods-principal component analysis (PCA) and probabilistic PCA (PPCA)-were compared with eight non-linear dimensionality reduction methods (kernel PCA, classical multi-dimensional scaling, isometric mapping, nearest neighbour estimation, stochastic neighbourhood embedding, autoencoder, stochastic proximity embedding, and Laplacian eigenmaps) on previously collected behavioural and EEG data from eight healthy non-sleep-deprived volunteers performing a 1D-visuomotor tracking task for 1 h. The effectiveness of the feature reduction algorithms was evaluated by visual inspection of class separation on 3D scatterplots, by trustworthiness scores, and by microsleep detection performance on a stacked-generalisation-based linear discriminant analysis (LDA) system estimating the microsleep/responsive state at 1 Hz based on the reduced features. On trustworthiness, PPCA outperformed PCA, but PCA outperformed all of the non-linear techniques. The trustworthiness score for each feature reduction method also correlated strongly with microsleep-state detection performance, providing strong validation of the ability of trustworthiness to estimate the relative effectiveness of feature reduction approaches, in terms of predicting performance, and ability to do so independently of the gold standard. Graphical abstract Proposed microsleep detection system.

Manifold Learning Algorithms Applied to Structural Damage Classification

A comparative study of four manifold learning algorithms was carried out to perform the dimensionality reduction process within a proposed methodology for damage classification in structural health monitoring (SHM). Isomap, locally linear embedding (LLE), stochastic proximity embedding (SPE), and laplacian eigenmaps were used as manifold learning algorithms. The methodology included several stages that comprised: data normalization, dimensionality reduction, classification through K-Nearest Neighbors (KNN) machine learning model and finally holdout cross-validation with 25% of data for training and the remaining 75% of data for testing. Results evaluated in an experimental setup showed that the best classification accuracy was 100% when the methodology uses isomap algorithm with a hyperparameter k of 170 and 8 dimensions as a feature vector at the input to the KNN classification machine.

Prognosis of Early Cervical Carcinoma Using Gene Expression Profiling

Cervical carcinoma remains a prime cause of cancer-related deaths in woman globally. Research into the prediction of the survivability for cervical cancer has been a challenge for researchers. Survival rates increase with earlier detection of cancer of the cervix. Cancer research and associated domains have made significant strides over recent years. For example, cancer prognosis using machine learning techniques is now a promising area of research. Data mining and machine learning have found considerable application thru the use of microarray expression profiling inspection. Specifically, DNA chip gene expression technology is a promising tool that can identify cancerous cells in early phases of the disease by examining the gene expression of analyzed instances. Furthermore, microarray technology enables researchers to assay the expression of thousands of genes in parallel. In this paper, we present a Gaussian process regression model in order improve the prediction of survivability of patients with early cervical cancer. Additionally, stochastic proximity embedding (SPE) was applied to reducing the number of attributes by selecting the most informative genes of the input dataset. Consequently, the computational complexity was reduced and the performance of the proposed model was increased. Our results indicate that gene expression profiles combined with carefully chosen learning algorithms can predict patient survival for certain diseases.

Linear and Nonlinear Kinematic Synergies in the Grasping Hand

Kinematic synergies in human hand movements have shown promising applications in dexterous control of robotic and prosthetic hands. We and others have previously derived kinematic synergies from human hand grasping movements using a widely used linear dimensionality reduction method, Principal Component Analysis (PCA). As the human biomechanical system is inherently nonlinear, using nonlinear dimensionality reduction methods to derive kinematic synergies might be expected to improve the representation of human hand movements in reduced dimensions. In this paper, we derived linear and nonlinear kinematic synergies from linear (PCA), globally nonlinear (Isomap, Stochastic Proximity Embedding (SPE), Sammon Mapping (SaM), and Stochastic Neighbor Embedding (SNE)) and locally nonlinear (Local Linear Embedding (LLE), LaplacianEigenmaps (LaE), and Local Tangent Space Alignment (LTSA)) dimensionality reduction methods. Synergies derived from linear PCA and nonlinear SaMwere able to capture multiple functional postures and physiological patterns. Results from natural hand grasping movements indicated that PCA performed better than all nonlinear dimensionality reduction methods used in the paper. Results from ASL postural movements indicated that PCA, SaM, and SPE better generalized over ASL postural movements when compared to other methods. Overall, our results show that PCA derived synergies offer qualitative and quantitative advantages over nonlinear methods as a limited number of kinematic synergies begin to be implemented in human prosthetics.

FSPE: Visualization of Hyperspectral Imagery Using Faithful Stochastic Proximity Embedding

Hyperspectral image visualization reduces color bands to three, but prevailing linear methods fail to address data characteristics, and nonlinear embeddings are computa- tionally demanding. Qualitative evaluation of embedding is also lacking. We propose faithful stochastic proximity embedding (FSPE), which is a scalable and nonlinear dimensionality re- duction method. FSPE considers the nonlinear characteristics of spectral signatures, yet it avoids the costly computation of geodesic distances that are often required by other nonlinear methods. Furthermore, we employ a pixelwise metric that measures the quality of hyperspectral image visualization at each pixel. FSPE outperforms the state-of-art methods by at least 12% on average and up to 25% in the qualitative measure. An implementation on graphics processing units is two orders of magnitude faster than the baseline. Our method opens the path to high-fidelity and real-time analysis of hyperspectral images.

Trustworthy dimension reduction for visualization different data sets

A new nonlinear dimension reduction (DR) method which is called Trustworthy Stochastic Proximity Embedding (TSPE) is introduced in this paper to visualize different types of data sets. TSPE overcome the main shortcomings of the DR by sending the false neighbour points to the correct locations, and preserving the neighbourhood relation to the true neighbours, which are inside the local neighbourhood. The visualization of our proposed method displays the trustworthy, useful and meaningful colours, where the objects of the image can be easily distinguished. The performances of TSPE and 20 dimension reduction methods are compared, and the efficiency of the proposed method in both visualization accuracy and computational cost is shown. The results showed the ability of our method in preserving neighbourhood relation, where they revealed more interested information. In real data set, the efficiency of the visualization of tensor images data sets by TSPE might help the specialist to make a good decision about a patient’s treatment. The comparison with experimental data set, as three dimensions of curved cylinder, showed the ability of TSPE to unfold this complex data set efficiently whilst preserving most information of the original data set.

Underwater localization using stochastic proximity embedding and multi-dimensional scaling

Localization for underwater acoustic sensor networks is an active research topic where a large number of techniques have been proposed recently. This paper addresses one of the open research issues, the impact of underwater sound speed variation on the localization accuracy. In this paper, modified versions of stochastic proximity embedding and multi-dimensional scaling localization algorithms customized for underwater application are proposed. The algorithms are found to provide good performance in underwater scenario as they take into account refractive ray bending of acoustic waves. Detailed study of the algorithm performance has been done and the results are reported. Cramer Rao Lower Bound for the problem is also derived.

From Virtual Screening to Bioactive Compounds by Visualizing and Clustering of Chemical Space.

Identification and visualization of ‘activity islands’ in chemical space is presented as a straightforward method for rapid automated identification of bioactive compounds and drug target profiling. We successfully applied this computational technique to finding inhibitors of Helicobacter pylori protease HtrA with new molecular scaffolds, and to deorphanizing of a compound from a combinatorial oxadiazole library. Bioactive molecules were discovered with minimal experimental effort. The results demonstrate that visualization of ‘chemical space’ provides an intuitive approach to molecular design and virtual screening in drug discovery, even in the absence of a three-dimensional receptor structure. Visualization of chemical data can help understand the structure of compound distributions in chemical space and guide molecular design experiments. [1–5] Commonly applied visualization techniques in chemistry are principal component analysis (PCA) [6] and self-organizing maps (SOMs, Kohonen networks). [7–9] Both methods have proven their value for visualization of compound libraries and virtual screening. Still, they suffer from several drawbacks. For example, a SOM’s quality to separate data depends on the chosen map size, i.e. the number of ‘neurons’ (local clusters, Voronoi fields), and determining the actual quality of a computed SOM projection is nontrivial. A perceived disadvantage is that SOMs lack immediate interpretability due to nonlinear projection. While for low-dimensional data linear projection by PCA seems to be preferable, SOMs have shown to produce more robust projections of high-dimensional data. [10] Here, we present a method for visualizing and interpreting high-dimensional chemical data, which is complementary to SOM and PCA projection and overcomes some of their disadvantages and limitations. The projection is based on stochastic proximity embedding (SPE). [11] SPE embeds data in a low-dimensional space in such a way that pairwise distances between compounds are preserved. As a consequence, patterns in the original high-dimensional data distribution become accessible to visual inspection. Making such patterns visible supports our intuitive interpretation how a molecular representation might distinguish between sets of compounds (e.g., active vs inactive) and create some kind of order in data space. Once activity islands are identified in the visualization, the compounds that form such local clusters can be extracted and subjected to biochemical tests. [12–15]

Stochastic Proximity Embedding on Graphics Processing Units: Taking Multidimensional Scaling to a New Scale

Stochastic proximity embedding (SPE) was developed as a method for efficiently calculating lower dimensional embeddings of high-dimensional data sets. Rather than using a global minimization scheme, SPE relies upon updating the distances of randomly selected points in an iterative fashion. This was found to generate embeddings of comparable quality to those obtained using classical multidimensional scaling algorithms. However, SPE is able to obtain these results in O(n) rather than O(n²) time and thus is much better suited to large data sets. In an effort both to speed up SPE and utilize it for even larger problems, we have created a multithreaded implementation which takes advantage of the growing general computing power of graphics processing units (GPUs). The use of GPUs allows the embedding of data sets containing millions of data points in interactive time scales.

Stochastic Proximity Embedding: Methods and Applications.

Since its inception in 1996, the stochastic proximity embedding (SPE) algorithm and its variants have been applied to a wide range of problems in computational chemistry and biology with notable success. At its core, SPE attempts to generate Euclidean coordinates for a set of points so that they satisfy a prescribed set of geometric constraints. The algorithm's appeal rests on three factors: 1) its conceptual and programmatic simplicity; 2) its superior speed and scaling properties; and 3) its broad applicability. Here, we review some of the key applications, outline known limitations and ways to circumvent them, and highlight additional problem domains where the use of this technique could lead to significant breakthroughs.

Distributed wireless sensor network localization using stochastic proximity embedding

In this paper, we propose a computationally efficient distributed Wireless Sensor Network (WSN) localization method based on Stochastic Proximity Embedding (SPE), which is a dimensionality reduction technique that finds a low dimensional embedding of a high dimensional data by preserving the pair-wise distance data information. Unlike the localization techniques based on classical Multidimensional Scaling (MDS), which is a popular dimensionality reduction technique, SPE method does not require a complete distance information matrix of the network and it scales linearly with the number of nodes in the network. Also the stochastic descent approach adopted in SPE provides an accurate position estimate in reasonable number of iterations. Through extensive simulation study of the proposed method, it is found to provide better results in both uniform and irregular shaped sensor networks.

Conformational Sampling with Stochastic Proximity Embedding and Self-Organizing Superimposition: Establishing Reasonable Parameters for Their Practical Use

Stochastic proximity embedding (SPE) and self-organizing superimposition (SOS) are two recently introduced methods for conformational sampling that have shown great promise in several application domains. Our previous validation studies aimed at exploring the limits of these methods and have involved rather exhaustive conformational searches producing a large number of conformations. However, from a practical point of view, such searches have become the exception rather than the norm. The increasing popularity of virtual screening has created a need for 3D conformational search methods that produce meaningful answers in a relatively short period of time and work effectively on a large scale. In this work, we examine the performance of these algorithms and the effects of different parameter settings at varying levels of sampling. Our goal is to identify search protocols that can produce a diverse set of chemically sensible conformations and have a reasonable probability of sampling biologically active space within a small number of trials. Our results suggest that both SPE and SOS are extremely competitive in this regard and produce very satisfactory results with as few as 500 conformations per molecule. The results improve even further when the raw conformations are minimized with a molecular mechanics force field to remove minor imperfections and any residual strain. These findings provide additional evidence that these methods are suitable for many everyday modeling tasks, both high- and low-throughput.

Conformational Analysis of Macrocycles: Finding What Common Search Methods Miss

As computational drug design becomes increasingly reliant on virtual screening and on high-throughput 3D modeling, the need for fast, robust, and reliable methods for sampling molecular conformations has become greater than ever. Furthermore, chemical novelty is at a premium, forcing medicinal chemists to explore more complex structural motifs and unusual topologies. This necessitates the use of conformational sampling techniques that work well in all cases. Here, we compare the performance of several popular conformational search algorithms on three broad classes of macrocyclic molecules. These methods include Catalyst, CAESAR, MacroModel, MOE, Omega, Rubicon and two newer self-organizing algorithms known as stochastic proximity embedding (SPE) and self-organizing superimposition (SOS) that have been developed at Johnson & Johnson. Our results show a compelling advantage for the three distance geometry methods (SOS, SPE, and Rubicon) followed to a lesser extent by MacroModel. The remaining techniques, particularly those based on systematic search, often failed to identify any of the lowest energy conformations and are unsuitable for this class of structures. Taken together with our previous study on drug-like molecules (Agrafiotis, D. K.; Gibbs, A.; Zhu, F.; Izrailev, S.; Martin, E. Conformational Sampling of Bioactive Molecules: A Comparative Study. J. Chem. Inf. Model., 2007, 47, 1067-1086), these results suggest that SPE and SOS are two of the most robust and universally applicable conformational search methods, with the latter being preferred because of its superior speed.

A self‐organizing algorithm for molecular alignment and pharmacophore development

We present a method for simultaneous three-dimensional (3D) structure generation and pharmacophore-based alignment using a self-organizing algorithm called Stochastic Proximity Embedding (SPE). Current flexible molecular alignment methods either start from a single low-energy structure for each molecule and tweak bonds or torsion angles, or choose from multiple conformations of each molecule. Methods that generate structures and align them iteratively (e.g., genetic algorithms) are often slow. In earlier work, we used SPE to generate good-quality 3D conformations by iteratively adjusting pairwise distances between atoms based on a set of geometric rules, and showed that it samples conformational space better and runs faster than earlier programs. In this work, we run SPE on the entire ensemble of molecules to be aligned. Additional information about which atoms or groups of atoms in each molecule correspond to points in the pharmacophore can come from an automatically generated hypothesis or be specified manually. We add distance terms to SPE to bring pharmacophore points from different molecules closer in space, and also to line up normal/direction vectors associated with these points. We also permit pharmacophore points to be constrained to lie near external coordinates from a binding site. The aligned 3D molecular structures are nearly correct if the pharmacophore hypothesis is chemically feasible; postprocessing by minimization of suitable distance and energy functions further improves the structures and weeds out infeasible hypotheses. The method can be used to test 3D pharmacophores for a diverse set of active ligands, starting from only a hypothesis about corresponding atoms or groups.

On the Effects of Permuted Input on Conformational Sampling of Drug‐like Molecules: an Evaluation of Stochastic Proximity Embedding

Conformational sampling is a problem of central importance in computer-aided drug design. A good conformational search method must not exhibit any intrinsic bias, and must provide confidence that important regions of conformational space are not missed during the search. A recent study by Carta et al. showed that this is not always the case, and that several popular conformational search methods, such as Omega, are very sensitive to the relative ordering of atoms and bonds in the connection table. Here, we examine the performance of a newer method known as stochastic proximity embedding, or SPE, using five diverse bioactive ligands extracted from the PDB. Our results confirm that the conformational ensembles produced by SPE using different permuted inputs are statistically indistinguishable, and well within the range of variability that would be expected from the stochastic nature of the method itself. This, along with the results of a more comprehensive comparative study (Agrafiotis et al., J. Chem. Info. Model, 2007, in press), provides further evidence that SPE is one of the most robust and competitive conformational search methods described to date.

Conformational Sampling of Bioactive Molecules: A Comparative Study

The necessity to generate conformations that sample the entire conformational space accessible to a given molecule is ubiquitous in the field of computer-aided drug design. Protein-ligand docking, 3D database searching, and 3D QSAR are three commonly used techniques that depend critically upon the quality and diversity of the generated conformers. Although there are a wide range of conformational search algorithms available, the extent to which they sample conformational space is often unclear. To address this question, we conducted a robust comparison of the search algorithms implemented in several widely used molecular modeling packages, including Catalyst, Macromodel, Omega, MOE, and Rubicon as well as our own method, stochastic proximity embedding (SPE). We found that SPE used in conjunction with conformational boosting, a heuristic for biasing conformational search toward more extended or compact geometries, along with Catalyst, are significantly more effective in sampling the full range of conformational space compared to the other methods, which show distinct preferences for either more extended or more compact geometries.

Conformational Boosting

Stochastic proximity embedding (SPE) is a novel self-organizing algorithm for sampling conformational space using geometric constraints derived from the molecular connectivity table. Here, we describe a simple heuristic that can be used in conjunction with SPE to bias the conformational search towards more extended or compact conformations, and thus greatly expand the range of geometries sampled during the search. The method uses a boosting strategy to generate a series of conformations, each of which is at least as extended (or compact) as the previous one. The approach is compared to several popular conformational sampling techniques using a reference set of 59 bioactive ligands extracted from the Protein Data Bank, and is shown to be significantly more effective in sampling the full range of molecular radii, with the exception of the Catalyst program, which was equally effective.

A method for quantifying and visualizing the diversity of QSAR models

Feature selection is one of the most commonly used and reliable methods for deriving predictive quantitative structure–activity relationships (QSAR). Many feature selection algorithms are stochastic in nature and often produce different solutions depending on the initialization conditions. Because some features may be highly correlated, models that are based on different sets of descriptors may capture essentially the same information, however, such models are difficult to recognize. Here, we introduce a measure of similarity between QSAR models that captures the correlation between the underlying features. This measure can be used in conjunction with stochastic proximity embedding (SPE) or multi-dimensional scaling (MDS) to create a meaningful visual representation of structure–activity model space and aid in the post-processing and analysis of results of feature selection calculations.

Exploring the nonlinear geometry of protein homology.

The explosion of biological data resulting from genomic and proteomic research has created a pressing need for data analysis techniques that work effectively on a large scale. An area of particular interest is the organization and visualization of large families of protein sequences. An increasingly popular approach is to embed the sequences into a low-dimensional Euclidean space in a way that preserves some predefined measure of sequence similarity. This method has been shown to produce maps that exhibit global order and continuity and reveal important evolutionary, structural, and functional relationships between the embedded proteins. However, protein sequences are related by evolutionary pathways that exhibit highly nonlinear geometry, which is invisible to classical embedding procedures such as multidimensional scaling (MDS) and nonlinear mapping (NLM). Here, we describe the use of stochastic proximity embedding (SPE) for producing Euclidean maps that preserve the intrinsic dimensionality and metric structure of the data. SPE extends previous approaches in two important ways: (1) It preserves only local relationships between closely related sequences, thus allowing the map to unfold and reveal its intrinsic dimension, and (2) it scales linearly with the number of sequences and therefore can be applied to very large protein families. The merits of the algorithm are illustrated using examples from the protein kinase and nuclear hormone receptor superfamilies.