Data Dimensionality Reduction Methods Research Articles

Artificial intelligence informed toxicity screening of amine chemistries used in the synthesis of hybrid organic–inorganic perovskites

AbstractThis article describes a machine learning guided framework for screening the potential toxicity impact of amine chemistries used in the synthesis of hybrid organic–inorganic perovskites. Using a combination of a probabilistic molecular fingerprint technique that encodes bond connectivity (MinHash) coupled to non‐linear data dimensionality reduction methods (Uniform Manifold Approximation and Projection), we develop an “Amine Atlas.” We show how the Amine Atlas can be used to rapidly screen the relative toxicity levels of amine molecules used in the synthesis of 2D and 3D perovskites and help identify safer alternatives. Our work also serves as a framework for rapidly identifying molecular similarity guided, structure–function relationships for safer materials chemistries that also incorporate sustainability/toxicity concerns.

Combining Spectral and Texture Features of UAS-Based Multispectral Images for Maize Leaf Area Index Estimation

The leaf area index (LAI) is of great significance for crop growth monitoring. Recently, unmanned aerial systems (UASs) have experienced rapid development and can provide critical data support for crop LAI monitoring. This study investigates the effects of combining spectral and texture features extracted from UAS multispectral imagery on maize LAI estimation. Multispectral images and in situ maize LAI were collected from test sites in Tongshan, Xuzhou, Jiangsu Province, China. The spectral and texture features of UAS multispectral remote sensing images are extracted using the vegetation indices (VIs) and the gray-level co-occurrence matrix (GLCM), respectively. Normalized texture indices (NDTIs), ratio texture indices (RTIs), and difference texture indices (DTIs) are calculated using two GLCM-based textures to express the influence of two different texture features on LAI monitoring at the same time. The remote sensing features are prescreened through correlation analysis. Different data dimensionality reduction or feature selection methods, including stepwise selection (ST), principal component analysis (PCA), and ST combined with PCA (ST_PCA), are coupled with support vector regression (SVR), random forest (RF), and multiple linear regression (MLR) to build the maize LAI estimation models. The results reveal that ST_PCA coupled with SVR has better performance, in terms of the VIs + DTIs (R2 = 0.876, RMSE = 0.239) and VIs + NDTIs (R2 = 0.877, RMSE = 0.236). This study introduces the potential of different texture indices for maize LAI monitoring and demonstrates the promising solution of using ST_PCA to realize the combining of spectral and texture features for improving the estimation accuracy of maize LAI.

Adaptive Data Dimensionality Reduction for Chemical Process Modeling Based on the Information Criterion Related to Data Association and Redundancy

Chemical process modeling is the basis for research and applications in related fields. With the development of industrial informatization, data-driven process modeling techniques are increasingly applied in chemical processes, helping to obtain more accurate results with less model development costs. However, due to the high-dimensional nonlinear characteristics of most chemical processes, problems such as the “curse of dimensionality” and information redundancy will render the models more prone to overfitting with reduced accuracies and weaker generalization abilities. Many data dimensionality reduction methods are adopted to mitigate the above problems, but most are limited by inaccurate association measurements and weak redundancy exclusion. In this paper, the extensive existence of data associations and information redundancies is first revealed through the analysis from an information-theoretic perspective. Then, a feature selection method based on conditional refined maximal information coefficient maximization (CRMICM) is proposed to improve the consistency of association measurement and the accuracy of redundancy estimation with limited samples. The final prediction modeling test for an actual fluidized catalytic cracking (FCC) process proves the extensive association between the variables and targets. Only a few variables are essential for the modeling, while the rest are redundant. Compared with other methods, CRMICM achieves the best dimensionality reduction effects on the FCC process data regarding the number of features and model prediction accuracy, showing its good applicability for chemical processes.

A comparative study of machine learning methods for predicting the evolution of brain connectivity from a baseline timepoint

BackgroundPredicting the evolution of the brain network, also called connectome, by foreseeing changes in the connectivity weights linking pairs of anatomical regions makes it possible to spot connectivity-related neurological disorders in earlier stages and detect the development of potential connectomic anomalies. Remarkably, such a challenging prediction problem remains least explored in the predictive connectomics literature. It is a known fact that machine learning (ML) methods have proven their predictive abilities in a wide variety of computer vision problems. However, ML techniques specifically tailored for the prediction of brain connectivity evolution trajectory from a single timepoint are almost absent. New methodTo fill this gap, we organized a Kaggle competition where 20 competing teams designed advanced machine learning pipelines for predicting the brain connectivity evolution from a single timepoint. The teams developed their ML pipelines with combination of data pre-processing, dimensionality reduction and learning methods. Each ML framework inputs a baseline brain connectivity matrix observed at baseline timepoint t0 and outputs the brain connectivity map at a follow-up timepoint t1. The longitudinal OASIS-2 dataset was used for model training and evaluation. Both random data split and 5-fold cross-validation strategies were used for ranking and evaluating the generalizability and scalability of each competing ML pipeline. ResultsUtilizing an inclusive approach, we ranked the methods based on two complementary evaluation metrics (mean absolute error (MAE) and Pearson Correlation Coefficient (PCC)) and their performances using different training and testing data perturbation strategies (single random split and cross-validation). The final rank was calculated using the rank product for each competing team across all evaluation measures and validation strategies. Furthermore, we added statistical significance values to each proposed pipeline. ConclusionIn support of open science, the developed 20 ML pipelines along with the connectomic dataset are made available on GitHub (https://github.com/basiralab/Kaggle-BrainNetPrediction-Toolbox). The outcomes of this competition are anticipated to lead the further development of predictive models that can foresee the evolution of the brain connectivity over time, as well as other types of networks (e.g., genetic networks).

Dynamic Process Monitoring Based on Integrated Statistic of Principal Component Analysis and Slow Feature Analysis

Process monitoring has attracted extensive interest for real-time operating evaluation due to the expectation of a safe and higher-quality production in chemical industry. Principal component analysis (PCA), an effective method for data dimensionality reduction, has been widely utilized in static process monitoring. However, industrial process data generally show dynamic characteristics because certain sequences of process variables are autocorrelated due to internal mechanisms. Recently, slow feature analysis (SFA) has been proved to have a good performance in extracting the slowly changing features in dynamic processes. To monitor both static and dynamic relations, a novel dynamic process monitoring method based on integrated statistic of PCA and SFA is proposed by extracting data feature of static and dynamic variables respectively. Variables are first grouped into static and dynamic categories according to their autocorrelation. For static part, PCA is applied to extract static variable features and calculate T2 and SPE statistics, while SFA is adopted in dynamic part for the extraction of dynamic variable features, and T2, Te2, S2 and Se2 statistics can be calculated accordingly. At last, these statistics can be integrated by a support vector data description (SVDD) to give a final process monitoring result. The performance of the proposed method is compared with other dynamic process monitoring methods on the benchmark Tennessee Eastman process (TEP).

Анализ существующих методов снижения размерности входных данных

The explosive growth of data arrays, both in the number of records and in attributes, has triggered the development of a number of platforms for handling big data (Amazon Web Services, Google, IBM, Infoworks, Oracle, etc.), as well as parallel algorithms for data analysis (classification, clustering, associative rules). This, in turn, has prompted the use of dimensionality reduction techniques. Feature selection, as a data preprocessing strategy, has proven to be effective and efficient in preparing data (especially high-dimensional data) for various data collection and machine learning tasks. Dimensionality reduction is not only useful for speeding up algorithm execution, but can also help in the final classification/clustering accuracy. Too noisy or even erroneous input data often results in less than desirable algorithm performance. Removing uninformative or low-informative columns of data can actually help the algorithm find more general areas and classification rules and generally achieve better performance. This article discusses commonly used data dimensionality reduction methods and their classification. Data transformation consists of two steps: feature generation and feature selection. A dis tinction is made between scalar feature selection and vector methods (wrapper methods, filtering methods, embedded methods and hybrid methods). Each method has its own advantages and disadvantages, which are outlined in the article. It describes the application of one of the most effective methods of dimensionality reduction - the method of correspondence analysis for CSE-CIC-IDS2018 dataset. The effectiveness of this method in the tasks of dimensionality reduction of the specified dataset in the detection of computer attacks is checked.

A generalized l 2,p-norm regression based feature selection algorithm

Feature selection is an important data dimension reduction method, and it has been used widely in applications involving high-dimensional data such as genetic data analysis and image processing. In order to achieve robust feature selection, the latest works apply the or -norm of matrix to the loss function and regularization terms in regression, and have achieved encouraging results. However, these existing works rigidly set the matrix norms used in the loss function and the regularization terms to the same or -norm, which limit their applications. In addition, the algorithms for solutions they present either have high computational complexity and are not suitable for large data sets, or cannot provide satisfying performance due to the approximate calculation. To address these problems, we present a generalized -norm regression based feature selection (-RFS) method based on a new optimization criterion. The criterion extends the optimization criterion of (-RFS) when the loss function and the regularization terms in regression use different matrix norms. We cast the new optimization criterion in a regression framework without regularization. In this framework, the new optimization criterion can be solved using an iterative re-weighted least squares (IRLS) procedure in which the least squares problem can be solved efficiently by using the least square QR decomposition (LSQR) algorithm. We have conducted extensive experiments to evaluate the proposed algorithm on various well-known data sets of both gene expression and image data sets, and compare it with other related feature selection methods.

Dimension Reduction Analysis of Vowel Signal Data Based on Manifold Learning

Data dimension reduction is an important method to overcome dimension disaster and obtain as much valuable information as possible. Speech signal is a kind of non-stationary random signal with high redundancy, and proper dimension reduction methods are needed to extract and analyze the signal features efficiently in speech signal processing. Studies have shown that manifold structure exists in high-dimensional data. Manifold dimension reduction method aiming at discovering the intrinsic geometric structure of data may be more effective in dealing with practical problems. This paper studies a data dimension reduction method based on manifold learning and applies it to the analysis of vowel signals.

Machine learning based anomaly detection and diagnosis method of spinning equipment driven by spectrogram data

Anomaly detection of spinning equipment monitors the running state using spectrogram based on quality data of sliver or yarn, thus detecting worn components, equipment faults or incorrect drafting parameters. In order to improve the detection efficiency and accuracy, an assembling anomaly detection method for spectrogram data is proposed in this study that combines traditional spectrum analysis method and machine-learning method. Because of the characteristics of wavelength-spectrum data, this method integrates a data dimensionality reduction method based on extended variable selection, and an isolated forest anomaly detection method, to split out wavelength-spectrum data when a fault occurs. Experimental analysis in study cases from the drawing frame and spinning frame is given to verify the effectiveness of the proposed method, as well as to explain parameter selection for improving detection accuracy.

Improvement and Optimization of Feature Selection Algorithm in Swarm Intelligence Algorithm Based on Complexity

The swarm intelligence algorithm simulates the behavior of animal populations in nature and is a new type of intelligent solution that is different from traditional artificial intelligence. Feature selection is a very common data dimensionality reduction method, which requires us to select the feature subset with the best evaluation criteria from the original feature set. Feature selection, as an effective data processing method, has become a hot research topic in the fields of machine learning, pattern recognition, and data mining and has received extensive attention and attention. In order to verify the improvement effect of the feature selection algorithm based on the swarm intelligence algorithm on the data, this paper conducts experiments on six classes in the city’s first middle school with similar conditions. First, count the current situation of the students in the class, then divide them into classes, use different algorithms to teach them, and count the changes of the students after a period of teaching. The experiment found that the performance of students under the feature selection algorithm is about 30% higher than other teaching methods, and the awareness of cooperation between students reaches 0.8. It solves the contradiction between popularization and improvement and solves the problems of polarization and transformation of underachievers. The individuality of the algorithm has been fully utilized and developed. The test results show that the improved algorithm has faster convergence speed and higher solution accuracy, and the feature selection algorithm based on swarm intelligence algorithm can effectively improve the efficiency of the algorithm.

Evaluation of Different Models for Non-Destructive Detection of Tomato Pesticide Residues Based on Near-Infrared Spectroscopy.

In this study, the possibility of non-destructive detection of tomato pesticide residues was investigated using Vis/NIRS and prediction models such as PLSR and ANN. First, Vis/NIR spectral data from 180 samples of non-pesticide tomatoes (used as a control treatment) and samples impregnated with pesticide with a concentration of 2 L per 1000 L between 350–1100 nm were recorded by a spectroradiometer. Then, they were divided into two parts: Calibration data (70%) and prediction data (30%). Next, the prediction performance of PLSR and ANN models after processing was compared with 10 spectral preprocessing methods. Spectral data obtained from spectroscopy were used as input and pesticide values obtained by gas chromatography method were used as output data. Data dimension reduction methods (principal component analysis (PCA), Random frog (RF), and Successive prediction algorithm (SPA)) were used to select the number of main variables. According to the values obtained for root-mean-square error (RMSE) and correlation coefficient (R) of the calibration and prediction data, it was found that the combined model SPA-ANN has the best performance (RC = 0.988, RP = 0.982, RMSEC = 0.141, RMSEP = 0.166). The investigational consequences obtained can be a reference for the development of internal content of agricultural products, based on NIR spectroscopy.

Industrial Control Malicious Traffic Anomaly Detection System Based on Deep Autoencoder

Industrial control network is a direct interface between information system and physical control process. Due to the lack of authentication, encryption, and other necessary security protection designs, it has become the main target of malicious attacks under the trend of increasing openness. In order to protect the industrial control systems, we examine the detection of abnormal traffic in industrial control network and propose a method of detecting abnormal traffic in industrial control network based on autoencoder technology. What is more, a new deep autoencoder model was designed to reduce the dimensionality of traffic data in industrial control network. In this article, the Kullback–Leibler divergence was added to the loss function to improve the ability of feature extraction and the ability to recover raw data. Finally, this model was compared with the traditional data dimensionality reduction method (principal component analysis (PCA), independent component analysis, and singular value decomposition) on gas pipeline dataset. The results show that the approach designed in this article outperforms the three methods in different scenes in terms of f1 score.

Sparse robust graph-regularized non-negative matrix factorization based on correntropy.

Non-negative Matrix Factorization (NMF) is a popular data dimension reduction method in recent years. The traditional NMF method has high sensitivity to data noise. In the paper, we propose a model called Sparse Robust Graph-regularized Non-negative Matrix Factorization based on Correntropy (SGNMFC). The maximized correntropy replaces the traditional minimized Euclidean distance to improve the robustness of the algorithm. Through the kernel function, correntropy can give less weight to outliers and noise in data but give greater weight to meaningful data. Meanwhile, the geometry structure of the high-dimensional data is completely preserved in the low-dimensional manifold through the graph regularization. Feature selection and sample clustering are commonly used methods for analyzing genes. Sparse constraints are applied to the loss function to reduce matrix complexity and analysis difficulty. Comparing the other five similar methods, the effectiveness of the SGNMFC model is proved by selection of differentially expressed genes and sample clustering experiments in three The Cancer Genome Atlas (TCGA) datasets.

Supervised data transformation and dimensionality reduction with a 3-layer multi-layer perceptron for classification problems

The aim of data transformation is to transform the original feature space of data into another space with better properties. This is typically combined with dimensionality reduction, so that the dimensionality of the transformed space is smaller. A widely used method for data transformation and dimensionality reduction is Principal Component Analysis (PCA). PCA finds a subspace that explains most of the data variance. While the new PCA feature space has interesting properties, such as removing linear correlation, PCA is an unsupervised method. Therefore, there is no guarantee that the PCA feature space will be the most appropriate for supervised tasks, such as classification or regression. On the other hand, 3-layer Multi Layer Perceptrons (MLP), which are supervised methods, can also be understood as a data transformation carried out by the hidden layer, followed by a classification/regression operation performed by the output layer. Given that the hidden layer is obtained after a supervised training process, it can be considered that it is performing a supervised data transformation. And if the number of hidden neurons is smaller than the input, also dimensionality reduction. Despite this kind of transformation being widely available (any neural network package that allows access to the hidden layer weights can be used), no extensive experimentation on the quality of 3-layer MLP data transformation has been carried out. The aim of this article is to carry out this research for classification problems. Results show that, overall, this transformation offers better results than the PCA unsupervised transformation method.

Artificial Neural Network and Data Dimensionality Reduction Based on Machine Learning Methods for PMSM Model Order Reduction

The present paper targets a solution for permanent motor synchronous machine (PMSM) model order reduction (MOR) using artificial neural networks and machine learning techniques for data dimensionality reduction. The neural networks are trained using data obtained from a series of electromagnetic Finite Element Analysis (FEA), conducted in conditions imposed by the data dimensionality reduction method. The workflow proposed to build the PMSM MOR, starts with data generation, goes further to its post-processing, and finishes with the model training and experimental validation. In the study, data dimensionality reduction procedure (adaptive data generation) is performed to increase the computational efficiency, also maintaining the model accuracy. Different data reduction approaches are compared from the computational cost’s point of view and their ease of use. The obtained results are compared to those obtained from FEA seeking the best solution for building the dynamic model. The resulting ROM is included in a real-time control prototyping platform to characterize machine’s performances. The model accuracy and its usability are proved in a comparative analysis with simulated versus experimental measurements.

Research on Face Image Digital Processing and Recognition Based on Data Dimensionality Reduction Algorithm.

Because face recognition is greatly affected by external environmental factors and the partial lack of face information challenges the robustness of face recognition algorithm, while the existing methods have poor robustness and low accuracy in face image recognition, this paper proposes a face image digital processing and recognition based on data dimensionality reduction algorithm. Based on the analysis of the existing data dimensionality reduction and face recognition methods, according to the face image input, feature composition, and external environmental factors, the face recognition and processing technology flow is given, and the face feature extraction method is proposed based on nonparametric subspace analysis (NSA). Finally, different methods are used to carry out comparative experiments in different face databases. The results show that the method proposed in this paper has a higher correct recognition rate than the existing methods and has an obvious effect on the XM2VTS face database. This method not only improves the shortcomings of existing methods in dealing with complex face images but also provides a certain reference for face image feature extraction and recognition in complex environment.

Novel attribute selection technique for an efficient intrusion detection system

An intrusion detection system (IDS) is one of the popular countermeasures against network attacks or anomalies in the network. IDS requires data dimension reduction methods such as data filtering, attribute selection, and data clustering to improve the system's performance. Identifying the most significant attributes will produce the computational time complexity drastically. This paper proposes novel attribute selection methods for data reduction. The NSL-KDD dataset is selected for experimental evaluation. The attribute value ratios (AVRs) is calculated using average values with numeric attributes and frequency with binary attributes to select attributes. The proposed attributes selection model is also compared with the traditional attribute section method that is correlation-based feature selection (CFS), information gain (IG) and gain ratio (GR). Further, the accuracy (ACC), detection rate (DR) and false alarm rate (FAR) are computed using the proposed attributes selection model with J48Consolidated classifier and achieve better results than other state-of-the-art methodologies.

An Efficient Network Security Situation Assessment Method Based on AE and PMU

Network security situation assessment (NSSA) is an important and effective active defense technology in the field of network security situation awareness. By analyzing the historical network security situation awareness data, NSSA can evaluate the network security threat and analyze the network attack stage, thus fully grasping the overall network security situation. With the rapid development of 5G, cloud computing, and Internet of things, the network environment is increasingly complex, resulting in diversity and randomness of network threats, which directly determine the accuracy and the universality of NSSA methods. Meanwhile, the indicator data is characterized by large scale and heterogeneity, which seriously affect the efficiency of the NSSA methods. In this paper, we design a new NSSA method based on the autoencoder (AE) and parsimonious memory unit (PMU). In our novel method, we first utilize an AE‐based data dimensionality reduction method to process the original indicator data, thus effectively removing the redundant part of the indicator data. Subsequently, we adopt a PMU deep neural network to achieve accurate and efficient NSSA. The experimental results demonstrate that the accuracy and efficiency of our novel method are both greatly improved.

Application of Data Dimensionality Reduction Methods to the Construction of Fuzzy Neural Networks

Многие задачи интеллектуального анализа данных могут быть сведены к задачам классификации и регрессии, современные подходы к решению которых основаны на применении нейронных сетей, деревьев решений и нечеткой логики, а также классических статистических методов. Решение сложных практических задач состоит из нескольких этапов, таких как выявление и отбор признаков, снижение размерности исходного пространства, поиск взаимосвязей и построение математической модели.В данной работе рассматривается задача снижения размерности данных при построении нечеткой нейронной сети с целью сокращения количества исходных признаков и повышения их информативности. Для ее решения рассмотрены несколько подходов: применение метода главных компонент, нейронной сети с архитектурой автокодировщика и ограниченной машины Больцмана. В отличие от метода главных компонент, применение автокодировщика и ограниченной машины Больцмана позволяет учесть нелинейные связи между имеющимися признаками. Важной особенностью предлагаемой в работе нечеткой нейронной сети, основанной на нечетких деревьях решений, является наглядность представления закономерностей в системе данных в виде логических правил с оценкой важности каждого правила. Применение представленных методов снижения размерности входных данных позволило существенно снизить погрешность аппроксимации. По результатам тестирования лучшей способностью обобщать данные среди рассмотренных методов обладает ограниченная машина Больцмана: ее применение для построения правил в нечеткой нейронной сети снижает погрешность аппроксимации в несколько раз по сравнению с методом главных компонент. Полученные в работе результаты могут быть использованы для построения универсальных нечетких аппроксиматоров при решении задач машинного обучения и анализа данных.

The data dimensionality reduction and bad data detection in the process of smart grid reconstruction through machine learning.

To detect false data injection attacks (FDIAs) in power grid reconstruction and solve the problem of high data dimension and bad abnormal data processing in the power system, thereby achieving safe and stable operation of the power grid system, this study introduces machine learning methods to explore the detection of FDIAs. First, through the utilization of the standard IEEE node system and the simulation of FDIAs under the condition of non-complete topology information, the construction of the attack data set is completed, and the MatPower tool is applied to simulate and analyze the data set. Second, based on the isolated Forest (iForest) abnormal score data processing algorithm combined with the Local Linear Embedding (LLE) data dimensionality reduction method, an algorithm for data feature extraction is constructed. Finally, based on the combination of the Convolutional Neural Network (CNN) and the Gated Recurrent Unit (GRU) network, an algorithm model for FDIAs detection is constructed. The results show that in the IEEE14-bus node and IEEE118-bus node systems, the overall distribution of the state estimated before and after the attack vector injection is consistent with the initial value. In the iFores algorithm, the number of iTree and the number of samples affect the extraction of abnormal score data. When the number of iTree n is determined to be 100, and the corresponding number of samples w is determined to be 10, the algorithm has the best detection effect. The FDIAs detection algorithm model based on CNN-GRU shows good detection effects under high attack intensity, with an accuracy rate of more than 95%, and its performance is better than other traditional detection algorithms. In this study, the bad data detection model based on deep learning has an active role in the realization of the safe and stable operation of the smart grid.