Problem Of High-dimensional Data Research Articles

Dimension Reduction With Prior Information for Knowledge Discovery.

This paper addresses the problem of mapping high-dimensional data to a low-dimensional space, in the presence of other known features. This problem is ubiquitous in science and engineering as there are often controllable/measurable features in most applications. To solve this problem, this paper proposes a broad class of methods, which is referred to as conditional multidimensional scaling (MDS). An algorithm for optimizing the objective function of conditional MDS is also developed. The convergence of this algorithm is proven under mild assumptions. Conditional MDS is illustrated with kinship terms, facial expressions, textile fabrics, car-brand perception, and cylinder machining examples. These examples demonstrate the advantages of conditional MDS over conventional dimension reduction in improving the estimation quality of the reduced-dimension space and simplifying visualization and knowledge discovery tasks. Computer codes for this work are available in the open-source cml R package.

Improving with Hybrid Feature Selection in Software Defect Prediction

Software defect prediction (SDP) is used to identify defects in software modules that can be a challenge in software development. This research focuses on the problems that occur in Particle Swarm Optimization (PSO), such as the problem of noisy attributes, high-dimensional data, and premature convergence. So this research focuses on improving PSO performance by using feature selection methods with hybrid techniques to overcome these problems. The feature selection techniques used are Filter and Wrapper. The methods used are Chi-Square (CS), Correlation-Based Feature Selection (CFS), and Forward Selection (FS) because feature selection methods have been proven to overcome data dimensionality problems and eliminate noisy attributes. Feature selection is often used by some researchers to overcome these problems, because these methods have an important function in the process of reducing data dimensions and eliminating uncorrelated attributes that can cause noisy. Naive Bayes algorithm is used to support the process of determining the most optimal class. Performance evaluation will use AUC with an alpha value of 0.050. This hybrid feature selection technique brings significant improvement to PSO performance with a much lower AUC value of 0.00342. Comparison of the significance of AUC with other combinations shows the value of FS PSO of 0.02535, CFS FS PSO of 0.00180, and CS FS PSO of 0.01186. The method in this study contributes to improving PSO in the SDP domain by significantly increasing the AUC value. Therefore, this study highlights the potential of feature selection with hybrid techniques to improve PSO performance in SDP.

Comparison of Genetic Algorithm and Recursive Feature Elimination on High Dimensional Data

The use of big data in companies is currently used in file processing. With large capacity files, it can affect the performance in terms of time in the company, so to overcome the problem of high-dimensional data, feature selection is used in selecting the number of features. On the WDC dataset with 30 features and 569 data points, feature selection is performed using the Recusive Feature Elimination (RFE) and Genetic Algorithm (GA) models. Then a comparison of evaluation values is made to determine which feature selection is best for solving the problem. From the 14 tables of evaluation results and discussion in tables 1 to 14, it is found that in the evaluation of accuracy and the use of weighted macros on precision, recall, and f1 score, using GA selection features has slightly higher results than RFE, so it is concluded that GA selection features are better at solving problems in high-dimensional data.

Causality-Based PCA Methods for Condition Modeling of Mechatronic Systems

This paper investigates the feature dimensionality reduction problem of high-dimensional data in condition modelling of complex mechatronic systems, aiming at improving performance, robustness and interpretability of classical principle component analysis (PCA) methods. First, the causal discovery algorithms are adopted to construct the corresponding causal network among monitoring variables. Then, two causality-based PCA methods are proposed, based on the corresponding qualitative reachability matrix and quantitative reachability strength matrix, respectively. Average treatment effect (ATE) values are considered as the quantitative measure of the causalities in a reachability strength matrix. The robustness and effectiveness of the proposed methods are verified on a synthetic dataset and four public datasets in comparison with two benchmark methods, i.e. PCA and autoencoder (AE). The proposed methods are also adopted in a high-speed train braking system for extracting informative features from the monitoring variables.

A stacked ensemble method based on TCN and convolutional bi-directional GRU with multiple time windows for remaining useful life estimation

With the widespread popularity of sensors, time-series data of engine degradation processes have been widely applied for remaining useful life (RUL) prediction. As a result, the large-dimensional, large-scale, and multi-state data characteristics of degraded data make accurate prediction very challenging. To overcome this issue, this paper proposes a stacked integration method based on temporal convolutional network (TCN) and convolutional bi-directional gate recurrent unit (CNN-Bi-GRU) with multiple time windows for RUL prediction, which has smaller ensemble dimensions and stronger reliability and adaptability. In the proposed model, the TCN model can well overcome the limitations of large amounts of data which leads to the problem of difficulty in learning temporal relationships; CNN-Bi-GRU model is used to extract important features to solve the problem of high-dimensional data. In addition, the multi-time window method is used to enhance the adaptability of the method and increase the information obtained by the model. Compared with the popular prediction methods, the prediction results of the proposed ensemble model have been improved by about 20 % on average on data sets. In addition, on the N-CMAPSS data set with larger data volume and more diverse states, the proposed model also achieves an average improvement of about 10% over comparison methods. It shows that the proposed method enhances the reliability and applicability of prediction.

Computer-Aided Diagnosis of Complications After Liver Transplantation Based on Transfer Learning.

Liver transplantation is one of the most effective treatments for acute liver failure, cirrhosis, and even liver cancer. The prediction of postoperative complications is of great significance for liver transplantation. However, the existing prediction methods based on traditional machine learning are often unavailable or unreliable due to the insufficient amount of real liver transplantation data. Therefore, we propose a new framework to increase the accuracy of computer-aided diagnosis of complications after liver transplantation with transfer learning, which can handle small-scale but high-dimensional data problems. Furthermore, since data samples are often high dimensional in the real world, capturing key features that influence postoperative complications can help make the correct diagnosis for patients. So, we also introduce the SHapley Additive exPlanation (SHAP) method into our framework for exploring the key features of postoperative complications. We used data obtained from 425 patients with 456 features in our experiments. Experimental results show that our approach outperforms all compared baseline methods in predicting postoperative complications. In our work, the average precision, the mean recall, and the mean F1 score reach 91.22%, 91.70%, and 91.18%, respectively.

Design of feature selection algorithm for high-dimensional network data based on supervised discriminant projection.

High dimension and complexity of network high-dimensional data lead to poor feature selection effect network high-dimensional data. To effectively solve this problem, feature selection algorithms for high-dimensional network data based on supervised discriminant projection (SDP) have been designed. The sparse representation problem of high-dimensional network data is transformed into an Lp norm optimization problem, and the sparse subspace clustering method is used to cluster high-dimensional network data. Dimensionless processing is carried out for the clustering processing results. Based on the linear projection matrix and the best transformation matrix, the dimensionless processing results are reduced by combining the SDP. The sparse constraint method is used to achieve feature selection of high-dimensional data in the network, and the relevant feature selection results are obtained. The experimental findings demonstrate that the suggested algorithm can effectively cluster seven different types of data and converges when the number of iterations approaches 24. The F1 value, recall, and precision are all kept at high levels. High-dimensional network data feature selection accuracy on average is 96.9%, and feature selection time on average is 65.1 milliseconds. The selection effect for network high-dimensional data features is good.

Augmented Industrial Data-Driven Modeling Under the Curse of Dimensionality

The curse of dimensionality refers to the problem of increased sparsity and computational complexity when dealing with high-dimensional data. In recent years, the types and variables of industrial data have increased significantly, making data-driven models more challenging to develop. To address this problem, data augmentation technology has been introduced as an effective tool to solve the sparsity problem of high-dimensional industrial data. This paper systematically explores and discusses the necessity, feasibility, and effectiveness of augmented industrial data-driven modeling in the context of the curse of dimensionality and virtual big data. Then, the process of data augmentation modeling is analyzed, and the concept of data boosting augmentation is proposed. The data boosting augmentation involves designing the reliability weight and actual-virtual weight functions, and developing a double weighted partial least squares model to optimize the three stages of data generation, data fusion, and modeling. This approach significantly improves the inter-pretability, effectiveness, and practicality of data augmentation in the industrial modeling. Finally, the proposed method is verified using practical examples of fault diagnosis systems and virtual measurement systems in the industry. The results demonstrate the effectiveness of the proposed approach in improving the accuracy and robustness of data-driven models, making them more suitable for real-world industrial applications.

Differential Privacy High-Dimensional Data Publishing Based on Feature Selection and Clustering

As a social information product, the privacy and usability of high-dimensional data are the core issues in the field of privacy protection. Feature selection is a commonly used dimensionality reduction processing technique for high-dimensional data. Some feature selection methods only process some of the features selected by the algorithm and do not take into account the information associated with the selected features, resulting in the usability of the final experimental results not being high. This paper proposes a hybrid method based on feature selection and a cluster analysis to solve the data utility and privacy problems of high-dimensional data in the actual publishing process. The proposed method is divided into three stages: (1) screening features; (2) analyzing the clustering of features; and (3) adaptive noise. This paper uses the Wisconsin Breast Cancer Diagnostic (WDBC) database from UCI’s Machine Learning Library. Using classification accuracy to evaluate the performance of the proposed method, the experiments show that the original data are processed by the algorithm in this paper while protecting the sensitive data information while retaining the contribution of the data to the diagnostic results.

A p-value based dimensionality reduction test for high dimensional means

With the rapid development of modern computing techniques, high-dimensional data are increasingly encountered in many studies. In this paper, we propose a three-step method to study the mean testing problem. The proposed test is based on the p-values calculated from the univariate tests and the dimension reduction method. Since it does not require explicit conditions of data dimension and sample size, we can use it to solve the mean testing problem of high-dimensional data, especially when the data dimension is much larger than the sample size. The new method can be implemented for the normal and non-normal distribution, which has a wide application. Various simulations are conducted to compare the testing power of the new method and the existing tests. The comparison shows that the new method has higher testing power. We also apply the proposed method to a real example of gene expression data.

A Review on Nature-Inspired Algorithms for Cancer Disease Prediction and Classification

In the era of healthcare and its related research fields, the dimensionality problem of high-dimensional data is a massive challenge as it is crucial to identify significant genes while conducting research on diseases like cancer. As a result, studying new Machine Learning (ML) techniques for raw gene expression biomedical data is an important field of research. Disease detection, sample classification, and early disease prediction are all important analyses of high-dimensional biomedical data in the field of bioinformatics. Recently, machine-learning techniques have dramatically improved the analysis of high-dimension biomedical data sets. Nonetheless, researchers’ studies on biomedical data faced the challenge of vast dimensions, i.e., the vast features (genes) with a very low sample space. In this paper, two-dimensionality reduction methods, feature selection, and feature extraction are introduced with a systematic comparison of several dimension reduction techniques for the analysis of high-dimensional gene expression biomedical data. We presented a systematic review of some of the most popular nature-inspired algorithms and analyzed them. The paper is mainly focused on the original principles behind each of the algorithms and their applications for cancer classification and prediction from gene expression data. Lastly, the advantages and disadvantages of nature-inspired algorithms for biomedical data are evaluated. This review paper may guide researchers to choose the most effective algorithm for cancer classification and prediction for the satisfactory analysis of high-dimensional biomedical data.

A Novel Deep Learning Representation for Industrial Control System Data

Feature extraction plays an important role in constructing artificial intelligence (AI) models of industrial control systems (ICSs). Three challenges in this field are learning effective representation from high-dimensional features, data heterogeneity, and data noise due to the diversity of data dimensions, formats and noise of sensors, controllers and actuators. Hence, a novel unsupervised learning autoencoder model is proposed for ICS data in this paper. Although traditional methods only capture the linear correlations of ICS features, our deep industrial representation learning model (DIRL) based on a convolutional neural network can mine high-order features, thus solving the problem of high-dimensional and heterogeneous ICS data. In addition, an unsupervised denoising autoencoder is introduced for noisy ICS data in DIRL. Training the denoising autoencoder allows the model to better mitigate the sensor noise problem. In this way, the representative features learned by DIRL could help to evaluate the safety state of ICSs more effectively. We tested our model with absolute and relative accuracy experiments on two large-scale ICS datasets. Compared with other popular methods, DIRL showed advantages in four common indicators of AI algorithms: accuracy, precision, recall, and F1-score. This study contributes to the effective analysis of large-scale ICS data, which promotes the stable operation of ICSs.

Precise identification of objects in a hyperspectral image by characterizing the distribution of pure signatures

<span lang="EN-US">Hyperspectral image (HSI) has been widely adopted in many real-world applications due to its potential to provide detailed information from spectral and spatial data in each pixel. However, precise classification of an object from HSI is challenging due to complex and highly correlated features that exhibit a nonlinear relationship between the acquired spectral unique to the HSI object. In literature, many research works have been conducted to address this problem. However, the problem of processing high-dimensional data and achieving the best resolution factor for any set of regions remains to be evolved with a suitable strategy. Therefore, the proposed study introduces simplified modeling of the hyperspectral image in which precise detection of regions is carried out based on the characterization of pure signatures based on the estimation of the maximum pixel mixing ratio. Moreover, the proposed system emphasizes the pixel unmixing problem, where input data is processed concerning wavelength computation, feature extraction, and hypercube construction. Further, a non-iterative matrix-based operation with a linear square method is performed to classify the region from the input hyperspectral image. The simulation outcome exhibits efficient and precise object classification is achieved by the proposed system in terms classified HSI object and processing time.</span>

A hybrid Artificial Immune optimization for high-dimensional feature selection

For high-dimensional data, the traditional feature selection method is slightly inadequate. At present, most of the existing hybrid search methods have problems of high computational cost and unsatisfactory feature reduction rate. In this paper, a hybrid feature selection method based on artificial immune algorithm optimization (HFSIA) is proposed to solve the feature reduction problem of high-dimensional data. This method combines the filter method with the metaheuristic-based search strategy more effectively. Inspired by biological research results, the method introduces a lethal mutation mechanism and a Cauchy mutation operator with adaptive adjustment factors to improve the search performance of the algorithm. In addition, this method introduces an adaptive adjustment factor in the population update stage to improve the problem of insufficient diversity of the original algorithm. The effective combination of these mechanisms enables the algorithm to obtain better search capability at a lower computational cost. Experimental comparisons with 23 state-of-the-art feature selection methods are conducted on 22 high-dimensional benchmark datasets. The results show that the computational cost of HFSIA is comparable to 5 classical feature selection methods known for their speed. Moreover, it achieves a higher average classification accuracy than 18 hybrid feature selection methods reported in the latest literature with the best feature reduction rate.

An Efficient Hybrid Feature Selection Method Using the Artificial Immune Algorithm for High-Dimensional Data.

Feature selection provides the optimal subset of features for data mining models. However, current feature selection methods for high-dimensional data also require a better balance between feature subset quality and computational cost. In this paper, an efficient hybrid feature selection method (HFIA) based on artificial immune algorithm optimization is proposed to solve the feature selection problem of high-dimensional data. The algorithm combines filter algorithms and improves clone selection algorithms to explore the feature space of high-dimensional data. According to the target requirements of feature selection, combined with biological research results, this method introduces the lethal mutation mechanism and the Cauchy operator to improve the search performance of the algorithm. Moreover, the adaptive adjustment factor is introduced in the mutation and update phases of the algorithm. The effective combination of these mechanisms enables the algorithm to obtain a better search ability and lower computational costs. Experimental comparisons with 19 state-of-the-art feature selection methods are conducted on 25 high-dimensional benchmark datasets. The results show that the feature reduction rate for all datasets is above 99%, and the performance improvement for the classifier is between 5% and 48.33%. Compared with the five classical filtering feature selection methods, the computational cost of HFIA is lower than the two of them, and it is far better than these five algorithms in terms of the feature reduction rate and classification accuracy improvement. Compared with the 14 hybrid feature selection methods reported in the latest literature, the average winning rates in terms of classification accuracy, feature reduction rate, and computational cost are 85.83%, 88.33%, and 96.67%, respectively.

A Novel Method Based on Nonlinear Binary Grasshopper Whale Optimization Algorithm for Feature Selection.

Feature Selection (FS) is considered as an important preprocessing step in data mining and is used to remove redundant or unrelated features from high-dimensional data. Most optimization algorithms for FS problems are not balanced in search. A hybrid algorithm called nonlinear binary grasshopper whale optimization algorithm (NL-BGWOA) is proposed to solve the problem in this paper. In the proposed method, a new position updating strategy combining the position changes of whales and grasshoppers population is expressed, which optimizes the diversity of searching in the target domain. Ten distinct high-dimensional UCI datasets, the multi-modal Parkinson's speech datasets, and the COVID-19 symptom dataset are used to validate the proposed method. It has been demonstrated that the proposed NL-BGWOA performs well across most of high-dimensional datasets, which shows a high accuracy rate of up to 0.9895. Furthermore, the experimental results on the medical datasets also demonstrate the advantages of the proposed method in actual FS problem, including accuracy, size of feature subsets, and fitness with best values of 0.913, 5.7, and 0.0873, respectively. The results reveal that the proposed NL-BGWOA has comprehensive superiority in solving the FS problem of high-dimensional data.

Unsupervised dimensionality reduction by jointing dynamic hypergraph and low-rank embedding for classification and clustering

Hypergraph learning is largely used to handle dimensional disaster problem of high-dimensional data in view of its flexible ability to model complex data correlations. However, the following three problems limit the performance of unsupervised hypergraph-based dimensionality reduction methods: (1) most of them are unable to obtain consistently good performance for different data sets; (2) noise or outliers in the data can lead to unreliability of the fixed hypergraph; and (3) the simple K-nearest neighbor strategy generates hypergraph with a fixed number of vertices in each hyperedge. To solve the above problems, an unsupervised dimensionality reduction method named joint dynamic hypergraph and low-rank embedding (DHLRE) is proposed. Specifically, DHLRE unifies hypergraph learning and low-rank learning into a single objective function. By doing so, both global and higher-order local structural information of the data can be extracted by DHLRE, and the two structural information complement each other to make the projection matrix have stronger discriminative power. In addition, the weight of hyperedge and hypergraph in DHLRE are dynamically generated in dimension-reduced space, which suppresses the ability of noise or outlier to affect the projection matrix. To divide hyperedge in a flexible way, a new approach without neighborhood parameter is proposed. By using this approach, the number of vertices of the hyperedge can be adaptively decided for different data sets. Further, an iterative algorithm is designed to solve the objective function of DHLRE. Finally, experimental results on several real-world data sets show that DHLRE outperforms others related methods for both classification and clustering tasks.

Construction of Artificial Intelligence Application Model for Supply Chain Financial Risk Assessment

An artificial intelligence integrated application model of supply chain financial risk assessment is constructed. Based on the financial data and supply chain data of listed companies in China’s new energy electric vehicle industry, the supply chain financial credit risk evaluation index system is constructed. The data samples are preprocessed by PCA as the input data of the support vector machine, which effectively solves the problem of high-dimensional data in supply chain finance. By improving the inertia weight of particle swarm optimization and introducing mutation operation, a dynamic mutation particle swarm optimization algorithm is proposed to avoid the problem of particles falling into a local minimum in the process of optimization. Finally, the improved optimization algorithm is used to optimize the parameters of SVM and input AdaBoost integration as a weak classifier to build an integrated model with good performance in many aspects. The model has been successfully applied to the credit risk assessment of China’s new energy vehicle supply chain finance. The comparison with other models shows that the constructed model has certain advantages in performance.

WRNFS: Width Residual Neuro Fuzzy System, a Fast-Learning Algorithm with High Interpretability

Although the deep neural network has a strong fitting ability, it is difficult to be applied to safety-critical fields because of its poor interpretability. Based on the adaptive neuro-fuzzy inference system (ANFIS) and the concept of residual network, a width residual neuro-fuzzy system (WRNFS) is proposed to improve the interpretability performance in this paper. WRNFS is used to transform a regression problem of high-dimensional data into the sum of several low-dimensional neuro-fuzzy systems. The ANFIS model in the next layer is established based on the low dimensional data and the residual of the ANFIS model in the former layer. The performance of WRNFS is compared with traditional ANFIS on three data sets. The results showed that WRNFS has high interpretability (fewer layers, fewer fuzzy rules, and fewer adjustable parameters) on the premise of satisfying the fitting accuracy. The interpretability, complexity, time efficiency, and robustness of WRNFS are greatly improved when the input number of single low-dimensional systems decreases.

An Ensemble Clustering Approach (Consensus Clustering) for High-Dimensional Data

Due to the plurality of irrelevant attributes, sparse distribution, and complicated calculations in high-dimensional data, traditional clustering algorithms, such as K-means, do not perform well on high-dimensional data. To address the clustering problem of high-dimensional data, this paper studies an integrated clustering method for high-dimensional data. A method of subspace division based on minimum redundancy is proposed to solve the problem of subspace division of high-dimensional data; subspace division is improved by using the K-means algorithm. Additionally, this method uses mutual information between the characteristic variables of the data to replace the calculation in the K-means algorithm. The distance between the characteristic variables of the data is used to divide the data into subspaces according to the mutual information values between the characteristic variables of the data. To achieve high clustering accuracy and diversity based on clustering requirements, this paper uses a genetic algorithm as the consistency integration function. The fitness function is designed according to the clustering fusion target, and the selection operator is designed according to the maximum number of overlapping elements in the base clustering. The experimental results show that the clustering algorithm proposed in this paper outperforms other methods on most datasets and is an effective clustering integration algorithm. The proposed clustering algorithm is compared with other commonly used clustering fusion algorithms on datasets to prove the advantages of the proposed algorithm.