Dimensionality Reduction Of Feature Space Research Articles

Improving Ultrasonic Testing by Using Machine Learning Framework Based on Model Interpretation Strategy

Ultrasonic testing (UT) is increasingly combined with machine learning (ML) techniques for intelligently identifying damage. Extracting significant features from UT data is essential for efficient defect characterization. Moreover, the hidden physics behind ML is unexplained, reducing the generalization capability and versatility of ML methods in UT. In this paper, a generally applicable ML framework based on the model interpretation strategy is proposed to improve the detection accuracy and computational efficiency of UT. Firstly, multi-domain features are extracted from the UT signals with signal processing techniques to construct an initial feature space. Subsequently, a feature selection method based on model interpretable strategy (FS-MIS) is innovatively developed by integrating Shapley additive explanation (SHAP), filter method, embedded method and wrapper method. The most effective ML model and the optimal feature subset with better correlation to the target defects are determined self-adaptively. The proposed framework is validated by identifying and locating side-drilled holes (SDHs) with 0.5λ central distance and different depths. An ultrasonic array probe is adopted to acquire FMC datasets from several aluminum alloy specimens containing two SDHs by experiments. The optimal feature subset selected by FS-MIS is set as the input of the chosen ML model to train and predict the times of arrival (ToAs) of the scattered waves emitted by adjacent SDHs. The experimental results demonstrate that the relative errors of the predicted ToAs are all below 3.67% with an average error of 0.25%, significantly improving the time resolution of UT signals. On this basis, the predicted ToAs are assigned to the corresponding original signals for decoupling overlapped pulse-echoes and reconstructing high-resolution FMC datasets. The imaging resolution is enhanced to 0.5λ by implementing the total focusing method (TFM). The relative errors of hole depths and central distance are no more than 0.51% and 3.57%, respectively. Finally, the superior performance of the proposed FS-MIS is validated by comparing it with initial feature space and conventional dimensionality reduction techniques.

Improving of the Generation Accuracy Forecasting of Photovoltaic Plants Based on <i>k</i>-Means and <i>k</i>-Nearest Neighbors Algorithms

Renewable energy sources (RES) are seen as a means of the fuel and energy complex carbon footprint reduction but the stochastic nature of generation complicates RES integration with electric power systems. Therefore, it is necessary to develop and improve methods for forecasting of the power plants generation using the energy of the sun, wind and water flows. One of the ways to improve the accuracy of forecast models is a deep analysis of meteorological conditions as the main factor affecting the power generation. In this paper, a method for adapting of forecast models to the meteorological conditions of photovoltaic stations operation based on machine learning algorithms was proposed and studied. In this case, unsupervised learning is first performed using the k-means method to form clusters. For this, it is also proposed to use studied the feature space dimensionality reduction algorithm to visualize and estimate the clustering accuracy. Then, for each cluster, its own machine learning model was trained for generation forecasting and the k-nearest neighbours algorithm was built to attribute the current conditions at the model operation stage to one of the formed clusters. The study was conducted on hourly meteorological data for the period from 1985 to 2021. A feature of the approach is the clustering of weather conditions on hourly rather than daily intervals. As a result, the mean absolute percentage error of forecasting is reduced significantly, depending on the prediction model used. For the best case, the error in forecasting of a photovoltaic plant generation an hour ahead was 9 %.

Calculation method of benchmark value of line loss rate in transformer district considering marketing customer portrait

In order to quantitatively analyze the benchmark value of line loss rate in the transformer district, understand the correlation and difference between samples in each transformer district, and improve the dynamic management ability of line loss in the transformer district, a method for calculating the benchmark value of line loss rate in the transformer district based on the portrait of marketing customers is proposed. The high-resolution feature quantity of the data in the transformer district is mined by the feature analysis method of marketing customer portrait, and the electrical parameters of the power grid, the operation mode of the power grid, the power flow distribution, the load and other factors are taken as the index parameters of the benchmark evaluation of the line loss rate in the transformer district, and the different characteristics of different stations are fully mined, and the static data and dynamic data of the transformer district are extracted respectively, and the abnormal data are corrected by the K nearest neighbor complementarity method. Combined with the characteristic differences of marketing customer portraits, the quantitative feature decomposition of the benchmark value of line loss rate in Transformer district is realized. According to the dynamic identification results of the benchmark value of line loss rate in Transformer district, the benchmark value of line loss rate in Transformer district with high-dimensional data is compressed into a low-dimensional space by using the feature space dimension reduction method, and the quantitative calculation of correlation and difference between samples in each Transformer district is realized. The simulation results show that the calculation complexity of this method is low, and the calculation method has a good ability to adjust the learning rate, ensuring a good local optimal solution in the learning process and high calculation accuracy.

Semi-Supervised Multi-Label Dimensionality Reduction Learning by Instance and Label Correlations

The label learning mechanism is challenging to integrate into the training model of the multi-label feature space dimensionality reduction problem, making the current multi-label dimensionality reduction methods primarily supervision modes. Many methods only focus attention on label correlations and ignore the instance interrelations between the original feature space and low dimensional space. Additionally, very few techniques consider how to constrain the projection matrix to identify specific and common features in the feature space. In this paper, we propose a new approach of semi-supervised multi-label dimensionality reduction learning by instance and label correlations (SMDR-IC, in short). Firstly, we reformulate MDDM which incorporates label correlations as a least-squares problem so that the label propagation mechanism can be effectively embedded into the model. Secondly, we investigate instance correlations using the k-nearest neighbor technique, and then present the l1-norm and l2,1-norm regularization terms to identify the specific and common features of the feature space. Experiments on the massive public multi-label data sets show that SMDR-IC has better performance than other related multi-label dimensionality reduction methods.

Motor Imagery BCI Classification with Frequency and Time-Frequency Features by Using Different Dimensions of the Feature Space Using Autoencoders

Brain-Computer Interfaces (BCIs) enable the users to directly communicate with machines based on various desired purposes through brain signals without moving any body parts. Thus, they have become very useful for prostheses, electric wheelchairs, virtual keyboards, and other studies like survey applications and emotion classifications. In this study, EEG signal processing was performed on the BCI Competition III-3a dataset, which contains motor imagery (MI) signals with four classes. Features of the non-stationary EEG signals belonging to three subjects were extracted using Power Spectral Density (PSD) with welch method, Wavelet Decomposition (WD), Empirical Mode Decomposition (EMD) and Hilbert-Huang Transform (HHT). From extracted 900 features, feature space dimension reduction was realized using Autoencoder, an unsupervised learning algorithm. The average accuracy obtained with Artificial Neural Network (ANN) is 74.5% for all binary classifications, which is generally a good result because of the non-stationary nature of EEG signals. 801 features yielded the best classification performance, obtained using an autoencoder with 400 hidden layer neurons.

Passive User Authentication Utilizing Behavioral Biometrics for IIoT Systems

Passive authentication is of great importance for security guarantee in industrial Internet of Things (IIoT) systems. Based on the behavioral biometrics from sequential operation actions in IIoT, this article proposes a nonintrusive and passive authentication framework for continuous user authentication against the impersonation attack. We first provide experimental results to demonstrate the discriminability and stability for the intrinsic features of sequential operation actions, and then leverage the Kalman filtering and wavelet techniques for noise elimination and the singular value decomposition method for the dimensionality reduction of feature space. We further exploit the one-class classification technique to formulate the authentication decision process as a hidden Markov model (HMM). Extensive experiments are conducted to illustrate the authentication performance of the passive authentication framework in terms of the false acceptance rate, false rejection rate, and equal-error rate. We also investigate the related authentication efficiency issues in terms of the usability to the operation-action sequence length, the scalability to the number of features and user space, and the sensitivity to the operation action features.

Improving Multi-Label Learning by Correlation Embedding

In multi-label learning, each object is represented by a single instance and is associated with more than one class labels, where the labels might be correlated with each other. As we all know, exploiting label correlations can definitely improve the performance of a multi-label classification model. Existing methods mainly model label correlations in an indirect way, i.e., adding extra constraints on the coefficients or outputs of a model based on a pre-learned label correlation graph. Meanwhile, the high dimension of the feature space also poses great challenges to multi-label learning, such as high time and memory costs. To solve the above mentioned issues, in this paper, we propose a new approach for Multi-Label Learning by Correlation Embedding, namely MLLCE, where the feature space dimension reduction and the multi-label classification are integrated into a unified framework. Specifically, we project the original high-dimensional feature space to a low-dimensional latent space by a mapping matrix. To model label correlation, we learn an embedding matrix from the pre-defined label correlation graph by graph embedding. Then, we construct a multi-label classifier from the low-dimensional latent feature space to the label space, where the embedding matrix is utilized as the model coefficients. Finally, we extend the proposed method MLLCE to the nonlinear version, i.e., NL-MLLCE. The comparison experiment with the state-of-the-art approaches shows that the proposed method MLLCE has a competitive performance in multi-label learning.

Hybrid Machine Learning for Solar Radiation Prediction in Reduced Feature Spaces

Solar radiation prediction is an important process in ensuring optimal exploitation of solar energy power. Numerous models have been applied to this problem, such as numerical weather prediction models and artificial intelligence models. However, well-designed hybridization approaches that combine numerical models with artificial intelligence models to yield a more powerful model can provide a significant improvement in prediction accuracy. In this paper, novel hybrid machine learning approaches that exploit auxiliary numerical data are proposed. The proposed hybrid methods invoke different machine learning paradigms, including feature selection, classification, and regression. Additionally, numerical weather prediction (NWP) models are used in the proposed hybrid models. Feature selection is used for feature space dimension reduction to reduce the large number of recorded parameters that affect estimation and prediction processes. The rough set theory is applied for attribute reduction and the dependency degree is used as a fitness function. The effect of the attribute reduction process is investigated using thirty different classification and prediction models in addition to the proposed hybrid model. Then, different machine learning models are constructed based on classification and regression techniques to predict solar radiation. Moreover, other hybrid prediction models are formulated to use the output of the numerical model of Weather Research and Forecasting (WRF) as learning elements in order to improve the prediction accuracy. The proposed methodologies are evaluated using a data set that is collected from different regions in Saudi Arabia. The feature-reduction has achieved higher classification rates up to 8.5% for the best classifiers and up to 15% for other classifiers, for the different data collection regions. Additionally, in the regression, it achieved improvements of average root mean square error up to 5.6% and in mean absolute error values up to 8.3%. The hybrid models could reduce the root mean square errors by 70.2% and 4.3% than the numerical and machine learning models, respectively, when these models are applied to some dataset. For some reduced feature data, the hybrid models could reduce the root mean square errors by 47.3% and 14.4% than the numerical and machine learning models, respectively.

A Gait Energy Image-Based System for Brazilian Sign Language Recognition

Sign language is the main type of communication of the deaf community. However, most people do not know this language, which causes communication problems for many people. Many technological solutions have been proposed to overcome this issue. Some use the concept of wearable devices, but gesture recognition in a video sequence is a cheaper and less intrusive solution. In this work, we address the problem of gesture recognition in video. To do so, we employ a two-step method with feature space mapping and classification. First, the body parts of each subject in a video are segmented through a deep neural network architecture. Then, we use Gait Energy Image to encode the motion of the body parts in a compact feature space. Small datasets are usually a problem in this type of application, leading to sparse representation in the feature space. To contour this problem, we evaluate SMOTE as a data augmentation technique in the feature space and classical dimensionality reduction techniques. We evaluate our method on three challenging Brazilian sign language (Libras) datasets, CEFET/RJ-Libras, MINDS-Libras, and LIBRAS-UFOP, achieving global accuracies of 85.40±3.13%, 84.66±1.78%, and 64.91±3.79%, respectively, with <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">singular value decomposition</i> and <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">support vector machine</i> .

Abstract 341: Predicting clinical endocrine response in advanced breast cancers using a reproducible low-dimensional biomarker

Abstract In the absence of reliable and effective prognostic biomarkers, endocrine therapy remains the standard of care for all advanced and metastatic estrogen receptor-positive (ER+) breast cancers. Attempts to develop biomarkers using the baseline tumor transcriptome or genome of advanced ER+ breast cancers have so far been unsuccessful due to predictive models with poor reproducibility in independent studies. Here we present an approach to develop a low-dimensional biomarker that estimates the risk of adverse events on endocrine therapy using the baseline tumor transcriptome of patients. Using a framework for supervised dimensionality reduction of the gene expression feature space, we constructed an endocrine response signature (ENDORSE) modeled on the survival outcomes of ER+ breast cancers from METABRIC. ENDORSE outperformed transcriptome-wide and knowledge-based signature models while significantly improving upon routine histopathological and genomic classifiers in cross-validation analyses. The ENDORSE risk estimate accurately predicted the outcomes for endocrine therapy in three independent clinical trials for ER+ breast cancers. Further, analysis of the phenotypes enriched in high-risk categories show endocrine resistance was not associated with rates of proliferation, but instead with a potential loss of DNA damage repair and cell-matrix interaction pathways. Citation Format: Aritro Nath, Adam Cohen, Jeffrey Chang, Andrea Bild. Predicting clinical endocrine response in advanced breast cancers using a reproducible low-dimensional biomarker [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 341.

Feature selection using hybrid poor and rich optimization algorithm for text classification

In order to reduce the high dimensional feature space in the text classification, feature selection plays a significant role. The dimension reduction of feature space reduces the computation cost and improves the text classification system accuracy. Hence, the identification of an optimal combination of features is an essential task in text classification. In this paper, the proposed work introduces a novel hybrid feature selection method based on binary poor and rich optimization algorithm (HBPRO) to obtain the appropriate subset of optimal features. The optimal feature subset which is selected by our proposed work is evaluated using Nave Bayes classifier with two popular benchmark text corpus datasets. The experimental results confirm that the proposed feature selection scheme (HBPRO) produces higher accuracy with a reduced number of features when compared with other feature selection techniques.

Independent role of CT chest scan in COVID-19 prognosis: Evidence from the machine learning classification

Background: The current coronavirus disease-19 (COVID-19) pandemic call attention to the key role informatics play in healthcare. The present study discovers an independent role of computerised tomography chest (CT) scans in prognosis of COVID-19 using classification learning algorithms. Methods: In this retrospective study, 57 RT PCR positive COVID-19 patients were enrolled from SMS Medical College, Jaipur (Rajasthan, India) after approval from the Institutional Ethics Committee. A set of 21 features including clinical findings and laboratory parameters and chest CT severity score were recorded. The CT score with mild, moderate and severe categories was chosen as response variable. The dimensionality reduction of feature space was performed and classifiers including, decision trees, K-nearest neighbours, support vector machine and ensemble learning were trained with principal components. The model with highest accuracy and area under the ROC curve (AUC) was selected. Results: The median age of patients was 55 years (range: 20-99 years) with 37 males. The feature space was reduced from 21 to 7 predictors, that included fever, cough, fibrin degradation products, haemoglobin, neutrophil-lymphocyte ratio, ferritin and procalcitonin. The linear support vector machine was chosen as the best classifier with 73.7 % and 0.69 accuracy and AUC for severe CT chest score, respectively. The variance contributed by first three principal components were 97.5 %, 2.4 % and 0.0 %, respectively. Conclusion: In view of low degree of relationships between predictors and chest CT scan severity score category as interpreted from accuracy and AUC it can be concluded that chest CT scan has an independent role in the prognosis of COVID-19 patients.

Application of linear discriminant analysis for classification of electrical grid emergency states

The approach of development of relay protection algorithms based on statistical data analysis suggests a transition from traditional protections to innovative multidimensional algorithms of recognition which are able to use the available information for emergency states identification more efficiently. The state parameters used for recognition are usually combined into a single multidimensional space. This results in computational complexity of the recognition algorithm. Thus, it is advisable to implement relay protection algorithms based on generalized features. This decision will make it possible to decrease the dimension of the feature space and increase response time of the protection. The linear discriminant analysis is used to reduce the feature space dimension when recognizing the states of an electrical grid section. The training sampling for statistical analysis is formed using a simulation model of the analyzed network section. The error matrix obtained as a result of the application of k-nearest neighbors algorithm for the state classification in the developed feature space was taken as a criterion for the feature space efficiency. A method of feature space dimension reduction for the task of multidimensional relay protection was proposed. It involves generalized features of response obtained via linear discriminant analysis. The method allowed formation of a setting plane which is the most effective for different types of short-circuit classification. This setting plane provides an error-free state classification while the setting planes obtained by pairs of initial features demonstrate an error of up to 50 %. The set goal of the feature space dimension reduction was achieved. Linear discriminant analysis is advisable to use by relay protection manufacturers for reduction of the feature space dimension in multidimensional relay protection problems, including multiple classification problems.

Acoustic-Based Engine Fault Diagnosis Using WPT, PCA and Bayesian Optimization

Engine fault diagnosis aims to assist engineers in undertaking vehicle maintenance in an efficient manner. This paper presents an automatic model and hyperparameter selection scheme for engine combustion fault classification, using acoustic signals captured from cylinder heads of the engine. Wavelet Packet Transform (WPT) is utilized for time–frequency analysis, and statistical features are extracted from both high- and low-level WPT coefficients. Then, the extracted features are used to compare three models: (i) standard classification model; (ii) Bayesian optimization for automatic model and hyperparameters selection; and (iii) Principle Component Analysis (PCA) for feature space dimensionality reduction combined with Bayesian optimization. The latter two models both demonstrated improved accuracy and the other performance metrics compared to the standard model. Moreover, with similar accuracy level, PCA with Bayesian optimized model achieved around 20% less total evaluation time and 8–19% less testing time, compared to the second model, for all fault conditions, which thus shows a promising solution for further development in real-time engine fault diagnosis.

Optimal Feature Subset Selection Based on Combining Document Frequency and Term Frequency for Text Classification

Feature selection plays a vital role to reduce the high dimension of the feature space in the text document classification problem. The dimension reduction of feature space reduces the computation cost and improves the text classification system accuracy. Hence, the identification of a proper subset of the significant features of the text corpus is needed to classify the data in less computational time with higher accuracy. In this proposed research, a novel feature selection method which combines the document frequency and the term frequency (FS-DFTF) is used to measure the significance of a term. The optimal feature subset which is selected by our proposed work is evaluated using Naive Bayes and Support Vector Machine classifier with various popular benchmark text corpus datasets. The experimental outcome confirms that the proposed method has a better classification accuracy when compared with other feature selection techniques.

Research on Software Defect Prediction Framework Based on ISFLA in IoT Communication Software

Software defect feature selection has problems of feature space dimensionality reduction and large search space. This research proposes a defect prediction feature selection framework based on improved shuffled frog leaping algorithm (ISFLA).Using the two-level structure of the framework and the improved hybrid leapfrog algorithm's own advantages, the feature values are sorted, and some features with high correlation are selected to avoid other heuristic algorithms in the defect prediction that are easy to produce local The case where the convergence rate of the optimal or parameter optimization process is relatively slow. The framework improves generalization of predictions of unknown data samples and enhances the ability to search for features related to learning tasks. At the same time, this framework further reduces the dimension of the feature space. After the contrast simulation experiment with other common defect prediction methods, we used the actual test data set to verify the framework for multiple iterations on Internet of Things (IoT) system platform. The experimental results show that the software defect prediction feature selection framework based on ISFLA is very effective in defect prediction of IoT communication software. This framework can save the testing time of IoT communication software, effectively improve the performance of software defect prediction, and ensure the software quality.

A Feature Selection Framework for Software Defect Prediction Using ISFLA

Aiming at the problem of feature space dimension reduction and large search space in feature selection of software defect, a defect prediction feature selection framework based on meta-heuristic search algorithm (ISFLA) is proposed. The framework improves generalization of predictions of unknown data samples, enhances the ability to search for features related to learning tasks, and completes further reductions in feature space dimensions. Using some NASA data sets, several common software defect prediction methods and ISFLA simulation experiments were carried out. The experimental results show that the software feature selection framework based on the improved shuffled frog leaping algorithm effectively improves the performance of software defect prediction.

Computational Identification of piRNAs Using Features Based on RNA Sequence, Structure, Thermodynamic and Physicochemical Properties.

Rationale PIWI-interacting RNAs (piRNAs) are a recently-discovered class of small non-coding RNAs (ncRNAs) with a length of 21-35 nucleotides. They play a role in gene expression regulation, transposon silencing, and viral infection inhibition. Once considered as “dark matter” of ncRNAs, piRNAs emerged as important players in multiple cellular functions in different organisms. However, our knowledge of piRNAs is still very limited as many piRNAs have not been yet identified due to lack of robust computational predictive tools.Methods To identify novel piRNAs, we developed piRNAPred, an integrated framework for piRNA prediction employing hybrid features like k-mer nucleotide composition, secondary structure, thermodynamic and physicochemical properties. A non-redundant dataset (D3349 or D1684p+1665n) comprising 1684 experimentally verified piRNAs and 1665 non-piRNA sequences was obtained from piRBase and NONCODE, respectively. These sequences were subjected to the computation of various sequence-structure based features in binary format and trained using different machine learning techniques, of which support vector machine (SVM) performed the best.Results During the ten-fold cross-validation approach (10-CV), piRNAPred achieved an overall accuracy of 98.60% with Mathews correlation coefficient (MCC) of 0.97 and receiver operating characteristic (ROC) of 0.99. Furthermore, we achieved a dimensionality reduction of feature space using an attribute selected classifier.Conclusion We obtained the highest performance in accurately predicting piRNAs as compared to the current state-of-the-art piRNA predictors. In conclusion, piRNAPred would be helpful to expand the piRNA repertoire, and provide new insights on piRNA functions.

Selecting informative features of human gene exons

Dimensionality reduction of the human gene exon feature space is considered with the aim of gene identification. To evaluate the performance of various feature selection algorithms, computational experiments were carried out using the examples of exons of 14 known human genes. It is proven that exons are clearly separable regarding gene affiliation. Feature selection algorithms are sensitive to noise features and allow to estimate their number. Reducing the number of features improves CPU-time, memory usage as well as reduces the complexity of a model and makes it easier to interpret. Our findings indicate that utilizing of features of flanking intronic sequences leads to better prediction models in comparison with utilizing of exon features. The results of the research provide new opportunities for study of human gene data using machine learning algorithms.

Visualizing histopathologic deep learning classification and anomaly detection using nonlinear feature space dimensionality reduction

BackgroundThere is growing interest in utilizing artificial intelligence, and particularly deep learning, for computer vision in histopathology. While accumulating studies highlight expert-level performance of convolutional neural networks (CNNs) on focused classification tasks, most studies rely on probability distribution scores with empirically defined cutoff values based on post-hoc analysis. More generalizable tools that allow humans to visualize histology-based deep learning inferences and decision making are scarce.ResultsHere, we leverage t-distributed Stochastic Neighbor Embedding (t-SNE) to reduce dimensionality and depict how CNNs organize histomorphologic information. Unique to our workflow, we develop a quantitative and transparent approach to visualizing classification decisions prior to softmax compression. By discretizing the relationships between classes on the t-SNE plot, we show we can super-impose randomly sampled regions of test images and use their distribution to render statistically-driven classifications. Therefore, in addition to providing intuitive outputs for human review, this visual approach can carry out automated and objective multi-class classifications similar to more traditional and less-transparent categorical probability distribution scores. Importantly, this novel classification approach is driven by a priori statistically defined cutoffs. It therefore serves as a generalizable classification and anomaly detection tool less reliant on post-hoc tuning.ConclusionRoutine incorporation of this convenient approach for quantitative visualization and error reduction in histopathology aims to accelerate early adoption of CNNs into generalized real-world applications where unanticipated and previously untrained classes are often encountered.