Efficient Dimensionality Reduction Research Articles

Sparse deep encoded features with enhanced sinogramic red deer optimization for fault inspection in wafer maps

AbstractDue to the complexity and dynamics of the semiconductor manufacturing processes, wafer bin maps (WBM) present various defect patterns caused by various process faults. The defect type detection on wafer maps provides information about the process and equipment in which the defect occurred. Recently, automatic inspection has played a vital role in meeting the high-throughput demand, especially with deep convolutional neural networks (DCNN) which shows promising efficiency. At the same time, the need for a large amount of labeled and balanced datasets limits the performance of such approaches. In addition, complex DCNN in recognition tasks can provide redundant features that cause overfitting and reduce interpretability. In this paper, a new hybrid deep model for wafer map fault detection to get over these challenges is proposed. Firstly, a new convolutional autoencoder (CAE) is employed as a synthetization model to fix the high imbalance problem of the dataset. Secondly, for efficient dimensionality reduction, an embedding procedure is applied to the synthesized maps to get sparse encoded wafer maps by reinforcing a sparsity regularization in an encoder-decoder network to form a sparsity-boosted autoencoder (SBAE). The sparse embedding of wafer maps guarantees more discriminative features with 50% reduction in spatial size compared to the original wafer maps. Then, the 2D encoded sparse maps are converted to 1D sinograms to be fed later into another aggressive feature reduction stage using a new modified red deer algorithm with a new tinkering strategy. The resultant feature pool is reduced to ~ 25 1D feature bases, i.e., ~ 1.5% of the initial size of the 2D wafer maps. Finally, for the prediction stage, a simple 1DCNN model is introduced. The proposed inspection model is tested via different experiments on real-world wafer map dataset (WM-811K). Compared to state-of-the-art techniques, the proposed model outperforms their performance even with small-sized 1D feature pool. The average testing accuracy are 98.77% and 98.8% for 9 and 8 types of faults, respectively.

A Local Approach to Parameter Space Reduction for Regression and Classification Tasks

Parameter space reduction has been proved to be a crucial tool to speed-up the execution of many numerical tasks such as optimization, inverse problems, sensitivity analysis, and surrogate models’ design, especially when in presence of high-dimensional parametrized systems. In this work we propose a new method called local active subspaces (LAS), which explores the synergies of active subspaces with supervised clustering techniques in order to carry out a more efficient dimension reduction in the parameter space. The clustering is performed without losing the input–output relations by introducing a distance metric induced by the global active subspace. We present two possible clustering algorithms: K-medoids and a hierarchical top–down approach, which is able to impose a variety of subdivision criteria specifically tailored for parameter space reduction tasks. This method is particularly useful for the community working on surrogate modelling. Frequently, the parameter space presents subdomains where the objective function of interest varies less on average along different directions. So, it could be approximated more accurately if restricted to those subdomains and studied separately. We tested the new method over several numerical experiments of increasing complexity, we show how to deal with vectorial outputs, and how to classify the different regions with respect to the LAS dimension. Employing this classification technique as a preprocessing step in the parameter space, or output space in case of vectorial outputs, brings remarkable results for the purpose of surrogate modelling.

A Graph-Based Mathematical Model for More Efficient Dimensionality Reduction of Landmark Data in Geometric Morphometrics

A Graph-Based Mathematical Model for More Efficient Dimensionality Reduction of Landmark Data in Geometric Morphometrics

A computationally efficient dimensionality reduction and attack classification approach for network intrusion detection

A computationally efficient dimensionality reduction and attack classification approach for network intrusion detection

GFLASSO-LR: Logistic Regression with Generalized Fused LASSO for Gene Selection in High-Dimensional Cancer Classification

Advancements in genomic technologies have paved the way for significant breakthroughs in cancer diagnostics, with DNA microarray technology standing at the forefront of identifying genetic expressions associated with various cancer types. Despite its potential, the vast dimensionality of microarray data presents a formidable challenge, necessitating efficient dimension reduction and gene selection methods to accurately identify cancerous tumors. In response to this challenge, this study introduces an innovative strategy for microarray data dimension reduction and crucial gene set selection, aiming to enhance the accuracy of cancerous tumor identification. Leveraging DNA microarray technology, our method focuses on pinpointing significant genes implicated in tumor development, aiding the development of sophisticated computerized diagnostic tools. Our technique synergizes gene selection with classifier training within a logistic regression framework, utilizing a generalized Fused LASSO (GFLASSO-LR) regularizer. This regularization incorporates two penalties: one for selecting pertinent genes and another for emphasizing adjacent genes of importance to the target class, thus achieving an optimal trade-off between gene relevance and redundancy. The optimization challenge posed by our approach is tackled using a sub-gradient algorithm, designed to meet specific convergence prerequisites. We establish that our algorithm’s objective function is convex, Lipschitz continuous, and possesses a global minimum, ensuring reliability in the gene selection process. A numerical evaluation of the method’s parameters further substantiates its effectiveness. Experimental outcomes affirm the GFLASSO-LR methodology’s high efficiency in processing high-dimensional microarray data for cancer classification. It effectively identifies compact gene subsets, significantly enhancing classification performance and demonstrating its potential as a powerful tool in cancer research and diagnostics.

Numerically stable locality-preserving partial least squares discriminant analysis for efficient dimensionality reduction and classification of high-dimensional data

Dimensionality reduction plays a pivotal role in preparing high-dimensional data for classification and discrimination tasks by eliminating redundant features and enhancing the efficiency of classifiers. The effectiveness of a dimensionality reduction algorithm hinges on its numerical stability. When data projections are numerically stable, they lead to enhanced class separability in the lower-dimensional embedding, consequently yielding higher classification accuracy. This paper investigates the numerical attributes of dimensionality reduction and discriminant subspace learning, with a specific focus on Locality-Preserving Partial Least Squares Discriminant Analysis (LPPLS-DA). High-dimensional data frequently introduce singularity in the scatter matrices, posing a significant challenge. To tackle this issue, the paper explores two robust implementations of LPPLS-DA. These approaches not only optimize data projections but also capture more discriminative features, resulting in a marked improvement in classification accuracy. Empirical evidence supports these findings through numerical experiments conducted on synthetic and spectral datasets. The results demonstrate the superior performance of the proposed methods when compared to several state-of-the-art dimensionality reduction techniques in terms of both classification accuracy and dimension reduction.

Fast Bayesian Linearized Inversion With an Efficient Dimension Reduction Strategy

Fast Bayesian Linearized Inversion With an Efficient Dimension Reduction Strategy

Sparse random hypergraphs: non-backtracking spectra and community detection

Abstract We consider the community detection problem in a sparse $q$-uniform hypergraph $G$, assuming that $G$ is generated according to the Hypergraph Stochastic Block Model (HSBM). We prove that a spectral method based on the non-backtracking operator for hypergraphs works with high probability down to the generalized Kesten–Stigum detection threshold conjectured by Angelini et al. (2015, Spectral detection on sparse hypergraphs. In: 2015 53rd Annual Allerton Conference on Communication, Control, and Computing (Allerton). IEEE, pp. 66–73). We characterize the spectrum of the non-backtracking operator for the sparse HSBM and provide an efficient dimension reduction procedure using the Ihara–Bass formula for hypergraphs. As a result, community detection for the sparse HSBM on $n$ vertices can be reduced to an eigenvector problem of a $2n\times 2n$ non-normal matrix constructed from the adjacency matrix and the degree matrix of the hypergraph. To the best of our knowledge, this is the first provable and efficient spectral algorithm that achieves the conjectured threshold for HSBMs with $r$ blocks generated according to a general symmetric probability tensor.

Deep neural network for learning wave scattering and interference of underwater acoustics

It is challenging to construct generalized physical models of underwater wave propagation owing to their complex physics and widely varying environmental parameters and dynamical scales. In this article, we present a deep convolutional recurrent autoencoder network (CRAN) for data-driven learning of complex underwater wave scattering and interference. We specifically consider the dynamics of underwater acoustic scattering from various non-uniform seamount shapes leading to complex wave interference patterns of back-scattered and forward-propagated waves. The CRAN consists of a convolutional autoencoder for learning low-dimensional system representation and a long short-term memory (LSTM)-based recurrent neural network for predicting system evolution in low dimensions. The convolutional autoencoder enables efficient dimension reduction of wave propagation by independently learning global and localized wave features. To improve the time horizon of wave dynamics prediction, we introduce an LSTM architecture with a single-shot learning mechanism and optimal time-delayed data embedding. On training the CRAN over 30 cases containing various seamount geometries and acoustic source frequencies, we can predict wave propagation up to a time horizon of 5 times the initiation sequence length for 15 out-of-training cases with a mean L2 error of approximately 10%. For selected out-of-training cases, the prediction time horizon could be increased to 6 times the initiation sequence length. Importantly, such predictions are obtained with physically consistent wave scattering and wave interference patterns and at 50% lower L2 error compared to routinely use standard LSTMs. These results demonstrate the potential of employing such deep neural networks for learning complex underwater ocean acoustic propagation physics.

Sparse robust adaptive unsupervised subspace learning for dimensionality reduction

This work is devoted to the investigation of dimension reduction problem. As an efficient dimension reduction method, much attention has been paid on unsupervised subspace learning since it does not rely on expensive labels. Firstly, we implant a robust estimator in the error term of objective function, this leads to that small coefficients can be automatically allocated to the abnormal points. Thus, our model is robust to noise and outliers. Posteriorly, the L2,r-norm (1≤r≤2) is used as a measure of error, then, the performance of the model can be improved by selecting the appropriate adaptive parameter r. Further, a L2,p-norm (0<p≤1) regularization term is added to the objective function, therefore the gained sparse subspace can further improve the efficiency and accuracy of the expression, as well as enhance the generalization ability of the model and reduce over-fitting. Moreover, an efficient algorithm with fast convergence speed is designed to solve the model. Finally, the experimental results on 14 datasets show that the subspace dimension obtained by our method is no more than 6. From the results of classification test, our algorithm has obvious advantages over the other similar six algorithms.

EEG based emotion recognition by hierarchical bayesian spectral regression framework

Spectral regression (SR), a graph-based learning regression model, can be used to extract features from graphs to realize efficient dimensionality reduction. However, due to the SR method remains a regularized least squares problem and being defined in L2-norm space, the effect of artifacts in EEG signals cannot be efficiently resisted. In this work, to further improve the robustness of the graph-based regression models, we propose to utilize the prior distribution estimation in the Bayesian framework and develop a robust hierarchical Bayesian spectral regression framework (named HB-SR), which is designed with the hierarchical Bayesian ensemble strategies. In the proposed HB-SR, the impact of noises can be effectively reduced by the adaptive adjustment approach in model parameters with the data-driven manner. Specifically, in the current work, three different distributions have been elaborately designed to enhance the universality of the proposed HB-SR, i.e., Gaussian distribution, Laplace distribution, and Student-t distribution. To objectively evaluate the performance of the HB-SR framework, we conducted both simulation studies and emotion recognition experiments based on emotional EEG signals. Experimental results have consistently indicated that compared with other existing spectral regression methods, the proposed HB-SR can effectively suppress the influence of noises and achieve robust EEG emotion recognition.

An efficient dimensionality reduction based on adaptive-GSM and transformer assisted classification for high dimensional data

An efficient dimensionality reduction based on adaptive-GSM and transformer assisted classification for high dimensional data

The motion reliability analysis of a series system with multiple coordinated manipulators

Coordinated manipulators can construct a series system, such that all the position error of the end-effector is required to remain within the permissible limit. This article proposed a new method to handle the problem of reliability analysis involving in multiple failure modes. The extreme value of the position error is introduced to construct a surrogate model to replace the original failure event, the moment with fraction order instead of the traditional integer is applied to derive the best-fit distribution of the extreme value based on the maximum entropy principle. The innovations mainly include: (1) the high efficient dimension reduction via the surrogate model constructed with the extreme value and (2) a better probability density function is derived through the fractional moment. A series system consists of three manipulators is used as an example to demonstrate the efficiency and accuracy of the proposed method.

Prognosis prediction of patients with malignant pleural mesothelioma using conditional variational autoencoder on 3D PET images and clinical data.

Deep learning (DL) has been widely used for diagnosis and prognosis prediction of numerous frequently occurring diseases. Generally, DL models require large datasets to perform accurate and reliable prognosis prediction and avoid overlearning. However, prognosis prediction of rare diseases is still limited owing to the small number of cases, resulting in small datasets. This paper proposes a multimodal DL method to predict the prognosis of patients with malignant pleural mesothelioma (MPM) with a small number of 3D positron emission tomography-computed tomography (PET/CT) images and clinical data. A 3D convolutional conditional variational autoencoder (3D-CCVAE), which adds a 3D-convolutional layer and conditional VAE to process 3D images, was used for dimensionality reduction of PET images. We developed a two-step model that performs dimensionality reduction using the 3D-CCVAE, which is resistant to overlearning. In the first step, clinical data were input to condition the model and perform dimensionality reduction of PET images, resulting in more efficient dimension reduction. In the second step, a subset of the dimensionally reduced features and clinical data were combined to predict 1-year survival of patients using the random forest classifier. To demonstrate the usefulness of the 3D-CCVAE, we created a model without the conditional mechanism (3D-CVAE), one without the variational mechanism (3D-CCAE), and one without an autoencoder (without AE), and compared their prediction results. We used PET images and clinical data of 520 patients with histologically proven MPM. The data were randomly split in a 2:1 ratio (train : test) and three-fold cross-validation was performed. The models were trained on the training set and evaluated based on the test set results. The area under the receiver operating characteristic curve (AUC) for all models was calculated using their 1-year survival predictions, and the results were compared. We obtained AUC values of 0.76 (95% confidence interval [CI], 0.72-0.80) for the 3D-CCVAE model, 0.72 (95% CI, 0.68-0.77) for the 3D-CVAE model, 0.70 (95% CI, 0.66-0.75) for the 3D-CCAE model, and 0.69 (95% CI 0.65-0.74) for the without AE model. The 3D-CCVAE model performed better than the other models (3D-CVAE, p=0.039; 3D-CCAE, p=0.0032; and without AE, p=0.0011). This study demonstrates the usefulness of the 3D-CCVAE in multimodal DL models learned using a small number of datasets. Additionally, it shows that dimensionality reduction via AE can be used to learn a DL model without increasing the overlearning risk. Moreover, the VAE mechanism can overcome the uncertainty of the model parameters that commonly occurs for small datasets, thereby eliminating the risk of overlearning. Additionally, more efficient dimensionality reduction of PET images can be performed by providing clinical data as conditions and ignoring clinical data-related features.

OCIF: automatically learning the optimized clinical information fusion method for computer-aided diagnosis tasks.

In computer-aided diagnosis, the fusion of image features extracted from neural networks and clinical information is crucial to improve diagnostic accuracy. How to integrate low-dimensional clinical information (LDCF) with high-dimensional network features (HDNF) is an urgent problem to be solved. We offer a new network search framework to address this problem, which can provide optimized LDCF fusion and efficient dimensionality reduction in HDNF. OCIF innovatively uses Gaussian process optimization to explore the search space for the number of fully connected (FC) layers, the number of neurons in each FC layer, the activation function, the dropout factor, and whether to add clinical information to each FC layer. Moreover, OCIF employs transfer learning to reduce the training parameter space and improve search efficiency. To evaluate the effectiveness of the proposed OCIF, we utilized three popular end-to-end overall survival (OS) time prediction models to predict the three classes. Our experimental results show that applying OCIF to a classical computer-aided diagnosis neural network can improve classification accuracy. Experiments on the 2020 BRATS dataset prove that OCIF achieves satisfactory performance, with an accuracy of 0.684, precision of 0.735, recall of 0.684, and F1-score of 0.675 on the OS time prediction task. OCIF effectively and creatively combines clinical information and network features, leveraging both clinical information and image features to enhance the accuracy of the final diagnosis. Our experiments demonstrate that the use of OCIF can significantly improve computer-aided diagnosis accuracy, and the approach has the potential to be extended to other medical classification tasks as well.

Multinomial Logistic Factor Regression for Multi-source Functional Block-wise Missing Data.

Multi-source functional block-wise missing data arise more commonly in medical care recently with the rapid development of big data and medical technology, hence there is an urgent need to develop efficient dimension reduction to extract important information for classification under such data. However, most existing methods for classification problems consider high-dimensional data as covariates. In the paper, we propose a novel multinomial imputed-factor Logistic regression model with multi-source functional block-wise missing data as covariates. Our main contribution is to establishing two multinomial factor regression models by using the imputed multi-source functional principal component scores and imputed canonical scores as covariates, respectively, where the missing factors are imputed by both the conditional mean imputation and the multiple block-wise imputation approaches. Specifically, the univariate FPCA is carried out for the observable data of each data source firstly to obtain the univariate principal component scores and the eigenfunctions. Then, the block-wise missing univariate principal component scores instead of the block-wise missing functional data are imputed by the conditional mean imputation method and the multiple block-wise imputation method, respectively. After that, based on the imputed univariate factors, the multi-source principal component scores are constructed by using the relationship between the multi-source principal component scores and the univariate principal component scores; and at the same time, the canonical scores are obtained by the multiple-set canonial correlation analysis. Finally, the multinomial imputed-factor Logistic regression model is established with the multi-source principal component scores or the canonical scores as factors. Numerical simulations and real data analysis on ADNI data show the proposed method works well.

A Statistical Downscaling Model Based on Multiway Functional Principal Component Analysis for Southern Australia Winter Rainfall

Abstract We propose a statistical downscaling model based on multiway functional principal component analysis (FPCA) for rainfall prediction. The model mainly explains the relationship between the winter mean sea level pressure (MSLP) and rainfall in southern Australia from the perspective of functional data. In comparison with the traditional approach of feature extraction based on principal component analysis, the multiway FPCA needs fewer principal components not only to capture the most variance in MSLP but also to greatly avoid the loss of spatial information. A functional principal component (FPC) regression is further developed to simulate both current and future rainfall. The main results show that the first five leading FPCs are sufficient to capture the spatial characteristics of winter MSLP, achieving the purpose of efficient dimensionality reduction. Specifically, no more than three FPCs are required to develop the functional downscaling models for the winter rainfall over four studied regions. The functional downscaling model provides a good skill in terms of the correlation higher than 0.7 between the predictions and observations and the ratio of root-mean-square error to the climatology of winter rainfall below 20% over four regions. The developed downscaling models are further used to interpret the MSLP patterns from four CMIP5 climate models [ACCESS1.3, BCC_CSM1.1(m), CESM1(CAM5), and MPI-ESM-MR], which have been used to simulate both present-day and future climate. The resulting downscaled values based on ensemble MSLP provide 1) a closer representation of observed present-day rainfall than the raw climate model values and 2) alternative estimates of future changes in rainfall that arise from changes in MSLP.

Application of Principal Component Analysis to advancing digital phenotyping of plant disease in the context of limited memory for training data storage

Despite its widespread employment as a highly efficient dimensionality reduction technique, limited research has been carried out on the advantage of Principal Component Analysis (PCA)–based compression/reconstruction of image data to machine learning-based image classification performance and storage space optimization. To address this limitation, we designed a study in which we compared the performances of two Convolutional Neural Network-Random Forest Algorithm (CNN-RF) guava leaf image classification models developed using training data from a number of original guava leaf images contained in a predefined amount of storage space (on the one hand), and a number of PCA compressed/reconstructed guava leaf images contained in the same amount of storage space (on the other hand), on the basis of four criteria – Accuracy, F1-Score, Phi Coefficient and the Fowlkes–Mallows index. Our approach achieved a 1:100 image compression ratio (99.00% image compression) which was comparatively much better than previous results achieved using other algorithms like arithmetic coding (1:1.50), wavelet transform (90.00% image compression), and a combination of three transform-based techniques – Discrete Fourier (DFT), Discrete Wavelet (DWT) and Discrete Cosine (DCT) (1:22.50). From a subjective visual quality perspective, the PCA compressed/reconstructed guava leaf images presented almost no loss of image detail. Finally, the CNN-RF model developed using PCA compressed/reconstructed guava leaf images outperformed the CNN-RF model developed using original guava leaf images by 0.10% accuracy increase, 0.10 F1-Score increase, 0.18 Phi Coefficient increase and 0.09 Fowlkes–Mallows increase.

Quantum dimensionality reduction by linear discriminant analysis

The dimensionality reduction is generally used as the data preprocessing stage of many machine learning tasks and has become an essential branch of information processing. Due to the exponential growth of data, how constructing reasonable and efficient dimensionality reduction algorithms has been an open challenge recently. In this paper, we propose a quantum linear discriminant analysis algorithm for dimensionality reduction. In the proposed algorithm, the idea of quantum solving linear equations is utilized to obtain the shadow principal components and an intermediate state. After that, the special unitary operations are designed to assist in completing the dimensionality reduction. The theoretical analysis shows the proposed algorithm has exponential acceleration on the number of vectors and a quadratic speedup on the dimensionality of the original data space. Since this algorithm solves the restriction that the between-class scatter matrix must be reversible, it makes the algorithm more applicable. Moreover, instead of getting the quantum state in the projection direction, the algorithm obtains the corresponding quantum state of dimensionality reduction data so that the outputs can be directly applied to other quantum machine learning algorithms. To illustrate it, a simple example, quantum K-nearest Neighbor classification, is provided.

Stabbing of Intrusion with Learning Framework Using Auto Encoder Based Intellectual Enhanced Linear Support Vector Machine for Feature Dimensionality Reduction

Using an Intelligent Intrusion Detection System (IIDS) instead of less effective firewalls and other intrusion detection systems can increase network security. The system's overall effectiveness is determined by the accuracy and speed of IIDS' categorization and training algorithms. According to research, Stabbing Intrusion Using Learning Framework (SILF) is an innovative and intelligent method of learning attack features and lowering dimensionality. To improve Enhanced Long Short-Term Memory (ELSTM) prediction accuracy while minimising testing and training time, an auto-encoder approach, which is an efficient learning methodology for feature generation in an unsupervised way is applied. Initial training samples are fed into the classifier to increase the predictability of incursion and classification accuracy. Thus, model efficacy may be achieved linearly while alternative classifier approaches such as conventional SVM, Random Forest (RF), and Naive Bayesian (NB) are investigated and compared. In this research, an efficient Intelligent Intrusion Detection System using Auto Encoder with Enhanced LSTM (IIDS-AE-ELSTM) is proposed for feature dimensionality reduction. Testing and training have shown that the proposed model is more efficient than existing systems in terms of performance measures such as accuracy, precision, recall, and F-measure. A new method to intrusion detection is presented, which increases detection of network intrusions with dimensionality reduction. The Python environment is used in the proposed model to create an efficient dimensionality reduction model for intrusion detection.