High-dimensional Data Space Research Articles

BSSA: Binary Salp Swarm Algorithm With Hybrid Data Transformation for Feature Selection

Feature selection is a technique commonly used in Data Mining and Machine Learning. Traditional feature selection methods, when applied to large datasets, generate a large number of feature subsets. Selecting optimal features within this high dimensional data space is time-consuming and negatively affects the system's performance. This paper proposes a new binary Salp Swarm Algorithm (bSSA) for selecting the best feature set from transformed datasets. The proposed feature selection method first transforms the original data-set using Principal Component Analysis (PCA) and fast Independent Component Analysis (fastICA) based hybrid data transformation methods; next, a binary Salp Swarm optimizer is used for finding the best features. The proposed feature selection approach improves accuracy and eliminates the selection of irrelevant features. We validate our technique on fifteen different benchmark data sets. We conduct an extensive study to measure the performance and feature selection accuracy of the proposed technique. The proposed bSSA is compared to Binary Genetic Algorithm (bGA), Binary Binomial Cuckoo Search (bBCS), Binary Grey Wolf Optimizer (bGWO), Binary Competitive Swarm Optimizer (bCSO), and Binary Crow Search Algorithm (bCSA). The proposed method attains a mean accuracy of 95.26% with 7.78% features on PCA-fastICA transformed datasets. The results show that bSSA outperforms the existing methods for the majority of the performance measures.

A Hybrid Method for Remaining Useful Life Prediction of Proton Exchange Membrane Fuel Cell Stack

Proton exchange membrane fuel cell (PEMFC) is a clean and efficient alternative technology for transport applications. The degradation analysis of the PEFMC stack plays a vital role in electric vehicles. We propose a hybrid method based on a deep neural network model, which uses the Monte Carlo dropout approach called MC-DNN and a sparse autoencoder model to analyze the power degradation trend of the PEMFC stack. The sparse autoencoder can map high-dimensional data space to low-dimensional latent space and significantly reduce noise data. Under static and dynamic operating conditions, using two experimental PEMFC stack datasets the predictive performance of our proposed model is compared with some published models. The results show that the MC-DNN model is better than other models. Regarding the remaining useful life (RUL) prediction, the proposed model can obtain more accurate results under different training lengths, and the relative error between 0.19% and 1.82%. In addition, the prediction interval of the predicted RUL is derived by using the MC dropout approach.

IEEE Access Special Section Editorial: Feature Representation and Learning Methods With Applications in Large-Scale Biological Sequence Analysis

Machine learning has been widely applied in the fields of biomedicine, computational biology, bioinformatics, image processing, and so on. The performance of machine learning methods mainly relies on feature representation that is the mapping from various types of raw data (i.e., image and genomic data) to a discriminant high-dimensional data space, bridging the raw data with the input of learning/inference algorithms. A good representation is often one that captures the discriminative information from the data and supports effective machine learning. However, over the last few decades, most representation learning approaches are labor-intensive and heavily dependent on the professional knowledge of researchers (dependent on handcrafted feature engineering). To conduct more novel applications in bioinformatics and biomedicine, the weakness of current learning algorithms should be overcome by developing novel feature representation learning algorithms, including supervised representation learning algorithms that are learning features from labeled data, unsupervised feature representation strategies that are learning feature representatives from unlabeled data, and deep feature representation learning algorithms that are learning representative features from data using deep learning architectures.

Effective Latent Representation for Prediction of Remaining Useful Life

AI approaches have been introduced to predict the remaining useful life (RUL) of a machine in modern industrial areas. To apply them well, challenges regarding the high dimension of the data space and noisy data should be met to improve model efficiency and accuracy. In this study, we propose an end-to-end model, termed ACB, for RUL predictions; it combines an autoencoder, convolutional neural network (CNN), and bidirectional long short-term memory. A new penalized root mean square error loss function is included to avoid an overestimation of the RUL. With the CNN-based autoencoder, a high-dimensional data space can be mapped into a lower-dimensional latent space, and the noisy data can be greatly reduced. We compared ACB with five state-of-the-art models on the Commercial Modular Aero-Propulsion System Simulation dataset. Our model achieved the lowest score value on all four sub-datasets. The robustness of our model to noise is also supported by the experiments.

Fault Diagnosis by Multisensor Data: A Data-Driven Approach Based on Spectral Clustering and Pairwise Constraints.

This paper deals with clustering based on feature selection of multisensor data in high-dimensional space. Spectral clustering algorithms are efficient tools in signal processing for grouping datasets sampled by multisensor systems for fault diagnosis. The effectiveness of spectral clustering stems from constructing an embedding space based on an affinity matrix. This matrix shows the pairwise similarity of the data points. Clustering is then obtained by determining the spectral decomposition of the Laplacian graph. In the manufacturing field, clustering is an essential strategy for fault diagnosis. In this study, an enhanced spectral clustering approach is presented, which is augmented with pairwise constraints, and that results in efficient identification of fault scenarios. The effectiveness of the proposed approach is described using a real case study about a diesel injection control system for fault detection.

Out-of-sample data visualization using bi-kernel t-SNE

T-distributed stochastic neighbor embedding (t-SNE) is an effective visualization method. However, it is non-parametric and cannot be applied to steaming data or online scenarios. Although kernel t-SNE provides an explicit projection from a high-dimensional data space to a low-dimensional feature space, some outliers are not well projected. In this paper, bi-kernel t-SNE is proposed for out-of-sample data visualization. Gaussian kernel matrices of the input and feature spaces are used to approximate the explicit projection. Then principal component analysis is applied to reduce the dimensionality of the feature kernel matrix. Thus, the difference between inliers and outliers is revealed. And any new sample can be well mapped. The performance of the proposed method for out-of-sample projection is tested on several benchmark datasets by comparing it with other state-of-the-art algorithms.

Performance Analysis of Intrusion Detection Systems Using a Feature Selection Method on the UNSW-NB15 Dataset

Computer networks intrusion detection systems (IDSs) and intrusion prevention systems (IPSs) are critical aspects that contribute to the success of an organization. Over the past years, IDSs and IPSs using different approaches have been developed and implemented to ensure that computer networks within enterprises are secure, reliable and available. In this paper, we focus on IDSs that are built using machine learning (ML) techniques. IDSs based on ML methods are effective and accurate in detecting networks attacks. However, the performance of these systems decreases for high dimensional data spaces. Therefore, it is crucial to implement an appropriate feature extraction method that can prune some of the features that do not possess a great impact in the classification process. Moreover, many of the ML based IDSs suffer from an increase in false positive rate and a low detection accuracy when the models are trained on highly imbalanced datasets. In this paper, we present an analysis the UNSW-NB15 intrusion detection dataset that will be used for training and testing our models. Moreover, we apply a filter-based feature reduction technique using the XGBoost algorithm. We then implement the following ML approaches using the reduced feature space: Support Vector Machine (SVM), k-Nearest-Neighbour (kNN), Logistic Regression (LR), Artificial Neural Network (ANN) and Decision Tree (DT). In our experiments, we considered both the binary and multiclass classification configurations. The results demonstrated that the XGBoost-based feature selection method allows for methods such as the DT to increase its test accuracy from 88.13 to 90.85% for the binary classification scheme.

Multi-level adaptive neuro-fuzzy inference system-based reconstruction of 1D ISOMAP representations

This paper addresses the inverse problem of isometric feature mapping (ISOMAP) via multi-level adaptive neuro-fuzzy inference system (ML-ANFIS). ISOMAP is a conventional nonlinear dimensionality reduction (NLDR) method, which prospects for low dimensional interior structure embedded in high dimensional data space. The inverse problem of ISOMAP reconstructs the original high dimensional data from the related low dimensional ISOMAP representations and holds promising applications in data representations, generation, compression and visualization. Because the reconstruction of 1D ISOMAP representations is ill-posed and undetermined, ML-ANFIS is wielded to augment the recovery quality of general ISOMAP reconstruction algorithm. By linearly combing inputs with nonlinear weights as output, ML-ANFIS can efficiently achieve the latent nonlinear relationship between the low-performance result of general ISOMAP reconstruction algorithm and its original data. The membership functions and fuzzy rules of ML-ANFIS can be automatically learned by gradient descent method as deep learning. It is demonstrated by the experimental results that, in the situation of 1D representations, the proposed method is superior to the state-of-the-art methods, such as nearest neighbor (NN), discrete cosine transformation (DCT), sparse representations (SR) and classical ANFIS algorithms, in the reconstruction performance of video data.

A hierarchical weighted low-rank representation for image clustering and classification

Low-rank representation (LRR), which is a powerful method to find the low-dimensional subspace structure embedded in high-dimensional data spaces, has been used in both unsupervised learning and semi-supervised classification. LRR aims at finding the lowest rank representation that can express each data sample as linear combination of other samples. However, this method doesn’t consider the geometrical structure of the data. Thus the similarity and local structure might be lost in the process of learning. Motivated by this, a novel hierarchical weighted low-rank representation (HWLRR) is proposed in this paper. In the new algorithm, a hierarchical weighted matrix is defined to find more samples that may belong to the same subspace using affinity propagation. By taking advantage of the affinity propagation, our proposed method can preserve both local and global structure of the whole dataset. The experimental results on both unsupervised learning and semi-supervised classification demonstrate the superiority of our proposed method.

A nonlinear method for monitoring industrial process

Aiming at fault detection in industrial processes with nonlinear or high dimensions, a novel method based on locally linear embedding preserve neighborhood for fault detection is proposed in this paper. Locally linear embedding preserve neighborhood is a feature-mapping method that combines Locally linear embedding and Laplacian eigenmaps algorithms. First, two weight matrices are obtained by the Locally linear embedding and Laplacian eigenmaps, respectively. Subsequently, the two weight matrices are combined by a balance factor to obtain the objective function. Locally linear embedding preserve neighborhood method can effectively maintain the characteristics of data in high-dimensional space. The purpose of dimension reduction is to map the high-dimensional data to low-dimensional space by optimizing the objective function. Process monitoring is performed by constructing T2 and Q statistics. To demonstrate its effectiveness and superiority, the proposed locally linear embedding preserve neighborhood for fault detection method is tested under the Swiss Roll dataset and an industrial case study. Compared with traditional fault detection methods, the proposed method in this paper effectively improves the detection rate and reduces the false alarm rate.

BLOCK-DBSCAN: Fast clustering for large scale data

We analyze the drawbacks of DBSCAN and its variants, and find the grid technique, which is used in Fast-DBSCAN and ρ-approximate DBSCAN, is almost useless in high dimensional data space. Because it usually yields considerable redundant distance computations. In order to tame these problems, two techniques are proposed: one is to use ϵ2-norm ball to identify Inner Core Blocks within which all points are core points, it has higher efficiency than grid technique for finding more core points at one time; the other is a fast approximate algorithm for judging whether two Inner Core Blocks are density-reachable from each other. Besides, cover tree is also used to accelerate the process of density computations. Based on the three techniques, an approximate approach, namely BLOCK-DBSCAN, is proposed for large scale data, which runs in about O(nlog (n)) expected time and obtains almost the same result as DBSCAN. BLOCK-DBSCAN has two versions, i.e., L2 version can work well for relatively high dimensional data, and L∞ version is suitable for high dimensional data. Experimental results show that BLOCK-DBSCAN is promising and outperforms NQDBSCAN, ρ-approximate DBSCAN and AnyDBC.

1D representation of Laplacian eigenmaps and dual k ‐nearest neighbours for unified video coding

This study proposes a framework of video coding based on Laplacian eigenmaps (LEM) and its related embedding and reconstruction algorithm (ERA). Firstly, a one-dimensional (1D) representation of LEM is adopted to achieve an extremely low bit per pixel (BPP). Secondly, dual k-nearest neighbours, which keeps neighbour relationships both in high-dimensional data space and low-dimensional representation space and overcomes the disadvantage of classical non-linear dimensionality reduction methods which cannot preserve the neighbour properties in both of the spaces, based ERA of LEM is employed to gain extraordinarily high peak-signal-to-noise ratio (PSNR). Thirdly, a unified framework of video coding is fit for intra-frame, inter-frame and multi-view video coding. Finally, it is evaluated by simulation experiments that, in the situation of low bitrate transmission, the proposed method can attain better performance of BPP and PSNR than that of the state-of-the-art methods, such as highly efficient video coding.

Sparse Graph Connectivity for Image Segmentation

It has been demonstrated that the segmentation performance is highly dependent on both subspace preservation and graph connectivity. In the literature, the full connectivity method linearly represents each data point ( e.g., a pixel in one image) by all data points for achieving subspace preservation, while the sparse connectivity method was designed to linearly represent each data point by a set of data points for achieving graph connectivity. However, previous methods only focused on either subspace preservation or graph connectivity. In this article, we propose a Sparse Graph Connectivity (SGC) method for image segmentation to automatically learn the affinity matrix from the low-dimensional space of original data, which aims at simultaneously achieving subspace preservation and graph connectivity. To do this, the proposed SGC simultaneously learns a self-representation affinity matrix for subspace preservation and a sparse affinity matrix for graph connectivity, from the intrinsic low-dimensional feature space of high-dimensional original data. Meanwhile, the self-representation affinity matrix is pushed to be similar to the sparse affinity as well as be the final segmentation results. Experimental result on synthetic and real-image datasets showed that our SGC method achieved the best segmentation performance, compared to state-of-the-art segmentation methods.

Traffic Incident Clearance Time Prediction and Influencing Factor Analysis Using Extreme Gradient Boosting Model

Accurate prediction and reliable significant factor analysis of incident clearance time are two main objects of traffic incident management (TIM) system, as it could help to relieve traffic congestion caused by traffic incidents. This study applies the extreme gradient boosting machine algorithm (XGBoost) to predict incident clearance time on freeway and analyze the significant factors of clearance time. The XGBoost integrates the superiority of statistical and machine learning methods, which can flexibly deal with the nonlinear data in high-dimensional space and quantify the relative importance of the explanatory variables. The data collected from the Washington Incident Tracking System in 2011 are used in this research. To investigate the potential philosophy hidden in data,K-means is chosen to cluster the data into two clusters. The XGBoost is built for each cluster. Bayesian optimization is used to optimize the parameters of XGBoost, and the MAPE is considered as the predictive indicator to evaluate the prediction performance. A comparative study confirms that the XGBoost outperforms other models. In addition, response time, AADT (annual average daily traffic), incident type, and lane closure type are identified as the significant explanatory variables for clearance time.

Spathial: an R package for the evolutionary analysis of biological data.

A primary problem in high-throughput genomics experiments is finding the most important genes involved in biological processes (e.g. tumor progression). In this applications note, we introduce spathial, an R package for navigating high-dimensional data spaces. spathial implements the Principal Path algorithm, which is a topological method for locally navigating on the data manifold. The package, together with the core algorithm, provides several high-level functions for interpreting the results. One of the analyses we propose is the extraction of the genes that are mainly involved in the progress from one state to another. We show a possible application in the context of tumor progression using RNA-Seq and single-cell datasets, and we compare our results with two commonly used algorithms, edgeR and monocle3, respectively. The R package spathial is available on the Comprehensive R Archive Network (https://cran.r-project.org/web/packages/spathial/index.html) and on GitHub (https://github.com/erikagardini/spathial). It is distributed under the GNU General Public License (version 3). Supplementary data are available at Bioinformatics online.

GLDH: Toward more efficient global low-density locality-sensitive hashing for high dimensions

Despite decades of intensive effort, the current solutions for efficiently searching high-dimensional data spaces are not entirely satisfactory. This paper proposes a more efficient global low-density locality sensitive hashing search algorithm (GLDH) based on the minimal cut hyperplane and ensemble learning. The innovation is that a novel global low-density hyperplane candidate set is constructed by the graph cut method, the minimum information gain method and random maximum entropy method are used to greedily select the hyperplane, and the ensemble learning method is used to query the global approximate nearest-neighbors data. This paper proves that the GLDH algorithm produces a low error hyperplane partition. The results of extensive experiments show that the proposed GLDH method performs better than the latest methods when using the same hash coding length for datasets from different fields.

SAMPLING OF SURFACES AND LEARNING FUNCTIONS IN HIGH DIMENSIONS.

The efficient representation of data in high-dimensional spaces is a key problem in several machine learning tasks. To capture the non-linear structure of the data, we model the data as points living on a smooth surface. We model the surface as the zero level-set of a bandlimited function. We show that this representation allows a non-linear lifting of the surface model, which will map the points to a low-dimensional subspace. This mapping between surfaces and the well-understood subspace model allows us to introduce novel algorithms (a) to recover the surface from few of its samples and (b) to learn a multidimensional bandlimited function from training data. The utility of these algorithms is introduced in practical applications including image denoising.

Representing Closed Transformation Paths in Encoded Network Latent Space

Deep generative networks have been widely used for learning mappings from a low-dimensional latent space to a high-dimensional data space. In many cases, data transformations are defined by linear paths in this latent space. However, the Euclidean structure of the latent space may be a poor match for the underlying latent structure in the data. In this work, we incorporate a generative manifold model into the latent space of an autoencoder in order to learn the low-dimensional manifold structure from the data and adapt the latent space to accommodate this structure. In particular, we focus on applications in which the data has closed transformation paths which extend from a starting point and return to nearly the same point. Through experiments on data with natural closed transformation paths, we show that this model introduces the ability to learn the latent dynamics of complex systems, generate transformation paths, and classify samples that belong on the same transformation path.

Information-Theoretic Based Clustering Method for High-Dimensional Data

Clustering is one of the important techniques for pattern recognition. Traditional clustering methods often group data objects by distance measurement. However, it is difficult to measure data in high-dimensional space. This paper proposes a clustering method for high-dimensional data. It combines the information theory criteria to establish clustering rules. The improved of K-Means is used to generate basis clustering, then the clustering ensemble is used to integrate the initial clusters to obtain a more stable final result. Our proposed method is tested on standard datasets, and its performance is compared with K-means ensemble. The experimental results indicate effectiveness for high-dimensional data clustering.

Research on Feature Extraction Method of Geometric Image Recognition

The existing image recognition feature extraction technology cannot effectively transmit the image information if the extracted features are not complete, sufficient and inaccurate. It will have an impact on image processing, such as reducing the accuracy and efficiency of subsequent image recognition and image tracking. At the same time, the existing technology for image processing destroys the topological structure between image pixels, and the process is complicated in the calculation of high-dimensional data space. Therefore, this paper proposes a method and steps of feature extraction for collective image recognition, and makes relevant supplementary explanation and analysis.