Structure Of High-dimensional Data Research Articles

Practical CO2—WAG Field Operational Designs Using Hybrid Numerical-Machine-Learning Approaches

Machine-learning technologies have exhibited robust competences in solving many petroleum engineering problems. The accurate predictivity and fast computational speed enable a large volume of time-consuming engineering processes such as history-matching and field development optimization. The Southwest Regional Partnership on Carbon Sequestration (SWP) project desires rigorous history-matching and multi-objective optimization processes, which fits the superiorities of the machine-learning approaches. Although the machine-learning proxy models are trained and validated before imposing to solve practical problems, the error margin would essentially introduce uncertainties to the results. In this paper, a hybrid numerical machine-learning workflow solving various optimization problems is presented. By coupling the expert machine-learning proxies with a global optimizer, the workflow successfully solves the history-matching and CO2 water alternative gas (WAG) design problem with low computational overheads. The history-matching work considers the heterogeneities of multiphase relative characteristics, and the CO2-WAG injection design takes multiple techno-economic objective functions into accounts. This work trained an expert response surface, a support vector machine, and a multi-layer neural network as proxy models to effectively learn the high-dimensional nonlinear data structure. The proposed workflow suggests revisiting the high-fidelity numerical simulator for validation purposes. The experience gained from this work would provide valuable guiding insights to similar CO2 enhanced oil recovery (EOR) projects.

Toward a Quantitative Survey of Dimension Reduction Techniques.

Dimensionality reduction methods, also known as projections, are frequently used in multidimensional data exploration in machine learning, data science, and information visualization. Tens of such techniques have been proposed, aiming to address a wide set of requirements, such as ability to show the high-dimensional data structure, distance or neighborhood preservation, computational scalability, stability to data noise and/or outliers, and practical ease of use. However, it is far from clear for practitioners how to choose the best technique for a given use context. We present a survey of a wide body of projection techniques that helps answering this question. For this, we characterize the input data space, projection techniques, and the quality of projections, by several quantitative metrics. We sample these three spaces according to these metrics, aiming at good coverage with bounded effort. We describe our measurements and outline observed dependencies of the measured variables. Based on these results, we draw several conclusions that help comparing projection techniques, explain their results for different types of data, and ultimately help practitioners when choosing a projection for a given context. Our methodology, datasets, projection implementations, metrics, visualizations, and results are publicly open, so interested stakeholders can examine and/or extend this benchmark.

Tensor Decomposition Learning for Compression of Multidimensional Signals

Multidimensional signals like multispectral images and color videos are becoming ubiquitous in modern times, constantly introducing challenges in data storage and transfer, and therefore demanding efficient compression strategies. Such high dimensional observations can be naturally encoded as tensors, exhibiting significant redundancies across dimensions. This property is exploited by tensor decomposition techniques that are being increasingly used for compactly encoding large multidimensional arrays. While efficient, these methods are incapable of utilizing prior information present in training data. In this paper, a novel tensor decomposition learning method is proposed for the compression of high dimensional signals. Specifically, instead of extracting independent bases for each example, our method learns an appropriate basis for each dimension from a set of training samples by solving a constrained optimization problem. As such, each sample is quantized and encoded into a reduced-size core tensor of coefficients that corresponds to the multilinear combination of the learned basis matrices. Furthermore, the proposed method employs a symbol encoding dictionary for binarizing the decomposition outputs. Experimental results on synthetic data and real satellite multispectral image sequences demonstrate the efficacy of our method, surpassing competing compression methods while offering the flexibility to handle arbitrary high dimensional data structures.

Deep reconstruction of 1D ISOMAP representations

This paper proposes a deep learning priors-based data reconstruction method of 1D isometric feature mapping (ISOMAP) representations. ISOMAP is a classical algorithm of nonlinear dimensionality reduction (NLDR) or manifold leaning (ML), which is devoted to questing for the low dimensional structure of high dimensional data. The reconstruction of ISOMAP representations, or the inverse problem of ISOMAP, reestablishes the high dimensional data from its low dimensional ISOMAP representations, and owns a bright future in data representation, generation, compression and visualization. Due to the fact that the dimension of ISOMAP representations is far less than that of the original high dimensional data, the reconstruction of ISOMAP representations is ill-posed or undetermined. Hence, the residual learning of deep convolutional neural network (CNN) is employed to boost reconstruction performance, via achieving the priors between the low-quality result of general ISOMAP reconstruction method and its residual relative to the original data. In the situation of 1D representations, it is evaluated by the experimental results that the proposed method outbalances the state-of-the-art methods, such as nearest neighbor (NN), discrete cosine transformation (DCT) and sparse representation (SR), in reconstruction performance of video data. In summary, the proposed method is suitable for low-bitrate and high-performance applications of data reconstruction.

Flexible Multi-View Unsupervised Graph Embedding

Faced with the increasing data diversity and dimensionality, multi-view dimensionality reduction has been an important technique in computer vision, data mining and multi-media applications. Since collecting labeled data is difficult and costly, unsupervised learning is of great significance. Generally, it is crucial to explore the complementarity or independence of different feature spaces in multi-view learning. How to find a low-dimensional subspace to preserve the intrinsic structure of original unlabeled high-dimensional multi-view data is still challenging. In addition, noises and outliers always appear in real data. In this study, we propose a novel model called flexible multi-view unsupervised graph embedding (FMUGE). A flexible regression residual term is introduced so that the strict linear mapping is relaxed, new-coming data and noises are better handled, and the raw data negotiate with the learned low-dimensional representation in the procedure. To ensure the consistency among multiple views, FMUGE adaptively weights different features and fuses them to get an optimal multi-view consensus similarity graph, which assists high-quality graph embedding. We propose an efficient alternating iterative algorithm to optimize the proposed model. Finally, experimental results on synthetic and benchmark datasets show the significant improvement of FMUGE over the state-of-the-art methods.

Unsupervised dimensionality reduction based on fusing multiple clustering results

The majority of the classical dimensionality reduction methods can be unified into a graph-embedding-based framework. A fixed graph constructed in a high-dimensional space has been extensively employed in the graph-embedding-based dimensionality reduction methods. However, a fixed graph often cannot characterize the structure of high-dimensional data owing to the curse of dimensionality. To solve this problem, we combine graph construction and dimensionality reduction into a coherent framework. Thus, the constructed graph can be updated dynamically in dimensionality reduction. In the existing methods based on the coherent framework, graphs are usually constructed by a type of neighborhood relationship and single clustering result. This study proposes an unsupervised dimensionality reduction method guided by fusing multiple clustering results. In the proposed method, multiple clustering results are first obtained by the k-means algorithm, and then a graph is constructed using a weighted co-association matrix of fusing the clustering results to capture data distribution information. Based on the graph, we present an objective function of combining graph construction and dimensionality reduction to implement mutual guidance between them. Numerical experiments on real data sets illustrate that the proposed method achieves significant improvement over some representative and state-of-the-art unsupervised dimensionality reduction methods.

Diffusion State Distances: Multitemporal Analysis, Fast Algorithms, and Applications to Biological Networks

Data-dependent metrics are powerful tools for learning the underlying structure of high-dimensional data. This article further develops and analyzes a data-dependent metric known as diffusion state distance (DSD), which compares points using a data-driven diffusion process. Unlike related diffusion methods, DSDs incorporate information across time scales, which allows for the intrinsic data structure to be inferred in a parameter-free manner. This article develops a theory for DSD based on the multitemporal emergence of mesoscopic equilibria in the underlying diffusion process. New algorithms for denoising and dimension reduction with DSD are also proposed and analyzed. These approaches are based on a weighted spectral decomposition of the underlying diffusion process, and experiments on synthetic datasets and real biological networks illustrate the efficacy of the proposed algorithms in terms of both speed and accuracy. Throughout, comparisons with related methods are made in order to illustrate the distinct advantages of DSD for datasets exhibiting multiscale structure.

Two Layer Tensor Form Convolutional Sparse Coding for Stereo Matching

Stereo matching is an important research topic in the field of computer vision. It recovers depth information from a pair of color images. Unfortunately, converting multi-dimensional (more than two-dimensional) data into two-dimensional data, such formulations ignore the spatial structure of multi-dimensional images/data. Tensors can be used to describe high-dimensional data structure, which can retain the hidden structure of data, but cannot obtain the deep features that helps to improve the performance of the algorithm. Therefore, it is very important to establish a deep tensor model. In this paper, we propose a two layer tensor form convolutional sparse coding model, which can automatically learn the deep convolutional kernel. Based on the learned two layer convolutional kernels, a two-layer dictionary learning model is established. Then, a new weighted matching cost method is constructed, which combines shallow and deep features. The experimental results on the Middlebury benchmark v2 and Middlebury benchmark v3 show that the proposed two layer tensor convolutional sparse coding is effective for stereo matching.

Mechanical Energy Expenditure at Lumbar Spine and Lower Extremity Joints During the Single-Leg Squat Is Affected by the Nonstance Foot Position.

Hirsch, SM, Chapman, CJ, Frost, DM, and Beach, TAC. Mechanical energy expenditure at lumbar spine and lower extremity joints during the single-leg squat is affected by the nonstance foot position. J Strength Cond Res 36(9): 2417-2426, 2022-Previous research has shown that discrete kinematic and kinetic quantities during bodyweight single-leg squat (SLS) movements are affected by elevated foot positioning and sex of the performer, but generalizations are limited by the high-dimensional data structure reported. Using a 3D inverse dynamical linked-segment model, we quantified mechanical energy expenditure (MEE) at each joint in the kinetic chain, the total MEE (sum of MEE across aforesaid joints), and the relative contribution of each joint to total MEE during SLSs performed with elevated foot positioned beside stance leg (SLS-Side), and in-front of (SLS-Front) and behind (SLS-Back) the body. Total MEE differed between SLS variations ( p = 0.002), with the least amount observed in the SLS-Back (effect size [ES] = 0.066-0.069). Approximately 50% of total MEE was contributed by the knee joint in each SLS variation, whereas MEE at the ankle, hip, and lumbar spine (in absolute and relative terms) varied complexly as a function of the elevated foot position. Total MEE ( p = 0.0192, ES = 0.852) and the absolute MEE at the knee and spine was greater in men across the SLS variations performed ( p = 0.025-0.036, ES = 0.715-0.766), but only the lumbar spine contribution to total MEE was larger in men across all SLS variations ( p = 0.045, ES = 0.607). Otherwise, there were no other sex-specific responses observed. Biomechanically, SLS movements are generally "knee-dominant," but changing elevated foot position effectively redistributes MEE among other joints in the linkage. Consistent with the previous conclusions reached based on discrete kinematic and kinetic data, not all SLSs are equal.

A Novel BN Learning Algorithm Based on Block Learning Strategy.

Learning accurate Bayesian Network (BN) structures of high-dimensional and sparse data is difficult because of high computation complexity. To learn the accurate structure for high-dimensional and sparse data faster, this paper adopts a divide and conquer strategy and proposes a block learning algorithm with a mutual information based K-means algorithm (BLMKM algorithm). This method utilizes an improved K-means algorithm to block the nodes in BN and a maximum minimum parents and children (MMPC) algorithm to obtain the whole skeleton of BN and find possible graph structures based on separated blocks. Then, a pruned dynamic programming algorithm is performed sequentially for all possible graph structures to get possible BNs and find the best BN by scoring function. Experiments show that for high-dimensional and sparse data, the BLMKM algorithm can achieve the same accuracy in a reasonable time compared with non-blocking classical learning algorithms. Compared to the existing block learning algorithms, the BLMKM algorithm has a time advantage on the basis of ensuring accuracy. The analysis of the real radar effect mechanism dataset proves that BLMKM algorithm can quickly establish a global and accurate causality model to find the cause of interference, predict the detecting result, and guide the parameters optimization. BLMKM algorithm is efficient for BN learning and has practical application value.

QUALITY ASSESSMENT OF DIMENSIONALITY REDUCTION TECHNIQUES ON HYPERSPECTRAL DATA: A NEURAL NETWORK BASED APPROACH

Abstract. Dimensionality reduction of hyperspectral images plays a vital role in remote sensing data analysis. The rapid advances in hyperspectral remote sensing has brought in a lot of opportunities to researchers to come up with advanced algorithms to analyse such voluminous data to better explore earth surface features. Modern machine learning algorithms can be applied to explore the underlying structure of high dimensional hyperspectral data and reduce the redundant information through feature extraction techniques. Limited studies have been carried out on dimensionality reduction for mineral exploration. The current study mainly focuses on the application of autoencoders for dimensionality reduction and provides a qualitative (visual) analysis of the obtained representations. The performance of autoencoders are investigated on Cuprite scene. Coranking matrix is used as evaluation criteria. From the obtained results it is evident that, deep autoencoders provide better results compared to single layer autoencoders. An increase in the number of hidden layers provides a better embedding. The neighborhood size K ≥ 40 of deep autoencoders provides a better transformation compared to autoencoders which shows an improved embedding only after K ≥ 80.

Calibration Transfer for Near-Infrared (NIR) Spectroscopy Based on Neighborhood Preserving Embedding

Calibration transfer is a subtle issue in the practical application of near-infrared (NIR) spectroscopy technique. In this paper, a novel method to calibration transfer based on neighborhood preserving embedding (CTNPE) for correcting spectral differences was proposed. As a manifold learning method, neighborhood preserving embedding (NPE) can not only capture the nonlinear manifold structure, but also retain the linearity and show good generalization ability. Since this approach can reveal low dimensional manifold structure in high dimensional spectroscopic data, it is beneficial to construct the transform relationship between source and target spectra. The performance of CTNPE was assessed and compared to that of piecewise direct standardization (PDS) and other four dimensionality reduction-based methods, including transfer based on target factor analysis (TTFA), spectral space transformation (SST), calibration transfer based on canonical correlation analysis (CTCCA) and based on independent component analysis (CTICA), in two real cases. The results indicated that CTNPE was able to successfully transfer spectra between instruments and samples in different physical states. Furthermore, CTNPE provided lower prediction errors than PDS, TTFA, CTCCA, SST, CTICA and direct prediction without a transfer function. Therefore, the comprehensive investigation carried out in the presented work demonstrates that CTNPE is a promising calibration transfer method for NIR, especially for correcting the variations for samples in different physical states.

Detecting changes in the second moment structure of high-dimensional sensor-type data in a K-sample setting

The K sample problem for high-dimensional vector time series is studied, especially focusing on sensor data streams, in order to analyze the second moment structure and detect changes across samples and/or across variables cumulated sum (CUSUM) statistics of bilinear forms of the sample covariance matrix. In this model, K independent vector time series are observed over a time span which may correspond to K sensors (locations) yielding d-dimensional data as well as K locations where d sensors emit univariate data. Unequal sample sizes are considered as arising when the sampling rate of the sensors differs. We provide large-sample approximations and two related change point statistics, a sum of squares and a pooled variance statistic. The resulting procedures are investigated by simulations and illustrated by analyzing a real data set.

Collaborative representation‐based locality preserving projections for image classification

Graph embedding has attracted much more research interests in dimensionality reduction. In this study, based on collaborative representation and graph embedding, the authors propose a new linear dimensionality reduction method called collaborative representation-based locality preserving projection (CRLPP). In the CRLPP, they assume that the similar samples should have similar reconstructions by collaborative representation and the similar reconstructions should also have the similar low-dimensional representations in the projected subspace. CRLPP first reconstructs each training sample using the collaborative representation of the other remaining training samples, and then designs the graph construction of all training samples, finally establishes the objective function of graph embedding using the collaborative reconstructions and the constructed graph. The proposed CRLPP can well preserve the intrinsic geometrical and discriminant structures of high-dimensional data in low-dimensional subspace. The effectiveness of the proposed is verified on several image datasets. The experimental results show that the proposed method outperforms the state-of-art dimensionality reduction.

Deep Double Supervised Embedding Neural Network Enhancing Class Separation for Visual High-Dimensional Industrial Process Monitoring

Visual process monitoring is the application of a visualization method to map the real-time operating information of an industrial process to a 2-D map, followed by process monitoring. However, owing to the complexity of industrial production processes and the complex correlations among industrial process variables, the structure and distribution of high-dimensional industrial data are very complicated. Therefore, a general visualization method cannot effectively separate the different fault data in a 2-D map for process monitoring. Accordingly, in this article, a deep double supervised embedding neural network (DDSE) is proposed for visualizing high-dimensional industrial data. The DDSE consists of two supervised deep neural networks: a deep class centres uniform distribution neural network (DCCUD), and a deep supervised t-stochastic neighbor embedding neural network (DSSNE). The DCCUD maps the high-dimensional industrial data to a new feature space in which the class centres obey a uniform distribution, promoting a good and separable situation for subsequent visualization procedures. The DSSNE then maps these high-dimensional features into a 2-D space. The training of the DDSE can be conducted through pre-training and fine-tuning. A proposed visual process monitoring approach combines the DDSE with the local outlier factor and k-nearest neighbor approaches. The proposed approach is tested on a Tennessee Eastman process, and the results illustrate that the proposed approach outperforms traditional methods in terms of visualization and visual process monitoring.

Stereo3D: using stereo images to enrich 3D visualization.

Visualization in 3D space is a standard but critical process for examining the complex structure of high-dimensional data. Stereoscopic imaging technology can be adopted to enhance 3D representation of many complex data, especially those consisting of points and lines. We illustrate the simple steps that are involved and strongly recommend others to implement it in designing visualization software. To facilitate its application, we created a new software that can convert a regular 3D scatterplot or network figure to a pair of stereo images. Stereo3D is freely available as an open source R package released under an MIT license at https://github.com/bioinfoDZ/Stereo3D. Others can integrate the codes and implement the method in academic software. deyou.zheng@einsteinmed.org. Supplementary data are available at Bioinformatics online.

Utilizing radar graphs in the visualization of simulation and estimation results in network meta-analysis.

Traditional visualization in meta-analysis uses forest plots to illustrate the combined treatment effect, along with the respective results from primary trials. While the purpose of visualization is clear in the pairwise setting, additional treatments broaden the focus and extend the results to be illustrated in network meta-analysis. The complexity increases further in situations where all potential contrasts in the network are compared to a predefined fixed value of interest, such as the 95% coverage evaluated against the nominal value of 95% in simulation studies. We propose utilizing radar graphs to illustrate results from network meta-analysis in cases where the interest lies in the comparison of estimated results (after fitting a network meta-analysis in a specific data set) or a performance measure (simulation study) to a pre-defined fixed reference value. Accounting for the complex high-dimensional data structure, the general picture of the full network is captured at once without increasing the space needed for visualization. Especially in large simulation studies, where multiple scenarios need to be visually combined to gain an overview on different scenarios, this type of illustration facilitates the discussion of results. Further properties, such as the expected variation due to the Monte-Carlo error or the differentiation between directly and indirectly estimated treatment contrasts in simulation studies, as well as the indication of well-connected and sparsely connected treatments in an applied network meta-analysis, can additionally be included in the visualization. While we used the radar-graph mainly for a simulation study, other applications are suitable whenever relative contributions of treatment (contrasts) are of interest.

Automated Detection of Surface Cracks and Numerical Correlation with Thermal-Structural Behaviors of Fire Damaged Concrete Beams

There are two specific aims in this study; first is to develop and validate an automated crack detection technique for the fire damaged beam. Second is to investigate whether the detected crack information and thermal-structural behaviors can be numerically related. To fulfill the aims, fire tests and residual strength tests are conducted on RC beams having different fire exposure time periods and sustained load levels. To detect the automated cracks, surface images of the fire damaged beam surfaces are taken with digital cameras and an automatic crack detection method is developed using a convolutional neural network (CNN) which is a deep learning technique primarily used for analyzing intricate structures of high-dimensional data [such as high definition (HD) images and videos]. The quantity of cracks detected using the proposed CNN changes depending on the test variables, and the changing trends are similar to those of the crack lengths obtained from the optical observation. Additionally, it is found that the quantity of the automatically detected cracks is numerically related to the temperatures inside the beams as well as the stiffnesses obtained from the residual strength tests.

Local Manifold-Based Sparse Discriminant Learning for Feature Extraction of Hyperspectral Image.

Hyperspectral image (HSI) generally contains a complex manifold structure and strong sparse correlation in its nonlinear high-dimensional data space. However, the existing manifold learning and sparse learning methods usually consider the manifold structure and sparse relationship separately rather than combining manifold and sparse properties to discover the intrinsic information in the original data. To simultaneously reveal the complex sparse relation and manifold structure of HSI, a novel feature extraction (FE) method, called local manifold-based sparse discriminant learning (LMSDL), has been proposed on the basis of manifold learning and sparse representation (SR). The LMSDL method first designs a new sparse optimization model called local manifold-based SR (LMSR) to reveal the local manifold-based sparse structure of data. Then, two geometrical sparse graphs are constructed to represent the discriminant relationship between samples and the geometrical and sparse neighbors. An objective function is constructed via geometrical sparse graphs and reconstruction points to learn a projection matrix for FE. The LMSDL effectively reveals the complex sparse relation and manifold structure in high-dimensional data, and it enhances the representation ability of extracted features for HSI classification significantly. The experimental results on the three real HSI datasets show that the proposed LMSDL algorithm possesses better performance in comparison with some state-of-the-art FE methods.

Vibration‐based detection and classification of structural changes using principal component analysis and ‐distributed stochastic neighbor embedding

This paper describes a structural health monitoring strategy to detect and classify structural changes in structures that can be equipped with sensors. The proposed approach is based on the t-distributed stochastic neighbor embedding (t-SNE), a nonlinear technique that can represent the local structure of high-dimensional data collected from multiple sensors in a plane or spatial representation. We propose the following basic steps for the detection and classification. First, the raw data are preprocessed: We scale the data using the mean-centered group scaling and apply principal component analysis to reduce the dimensionality of the scaled data. Second, t-SNE is applied to represent the scaled and reduced data as points in a plane, defining a cluster for each structural state. Finally, the current structure to be diagnosed is associated with a cluster (or structural state) using three different strategies: (a) the smallest point-centroid distance; (b) the majority voting; and (c) the sum of the inverse distances. The combination of t-SNE with our preprocessing and the three proposed classification strategies signif- icantly improves the quality of the clusters that represent different structural states. We evaluate the performance of our method using experimental data from an aluminum plate instrumented with piezoelectric transducers. Results are presented in the time domain, and they reveal the high classification accuracy and strong performance of this method, with a percentage of correct decisions close to 100% in several scenarios.