Data Dimensionality Reduction Research Articles

A hybrid quantum ensemble learning model for malicious code detection

Quantum computing as a new computing model with parallel computing capability and high information carrying capacity, has attracted a lot of attention from researchers. Ensemble learning is an effective strategy often used in machine learning to improve the performance of weak classifiers. Currently, the classification performance of quantum classifiers is not satisfactory enough due to factors such as the depth of quantum circuit, quantum noise, and quantum coding method, etc. For this reason, this paper combined the ensemble learning idea and quantum classifiers to design a novel hybrid quantum machine learning model. Firstly, we run the Stacking method in classical machine learning to realize the dimensionality reduction of high-latitude data while ensuring the validity of data features. Secondly, we used the Bagging method and Bayesian hyperparameter optimization method applied to quantum support vector machine (QSVM), quantum K nearest neighbors (QKNN), variational quantum classifier (VQC). Thirdly, the voting method is used to ensemble the predict results of QSVM, QKNN, VQC as the final result. We applied the hybrid quantum ensemble machine learning model to malicious code detection. The experimental results show that the classification precision (accuracy, F1-score) of this model has been improved to 98.9% (94.5%, 94.24%). Combined with the acceleration of quantum computing and the higher precision rate, it can effectively deal with the growing trend of malicious codes, which is of great significance to cyberspace security.

Machine Learning and Feature Selection for Soil Spectroscopy. An Evaluation of Random Forest Wrappers to Predict Soil Organic Matter, Clay, and Carbonates

Soil spectroscopy estimates soil properties using the absorption features in soil spectra. However, modelling soil properties with soil spectroscopy is challenging due to the high dimensionality of spectral data. Feature Selection wrapper methods are promising approaches to reducing dimensionality but are barely used in soil spectroscopy. The aim of this study is to evaluate the performance of two feature selection wrapper methods, Sequential Forward Selection (SFS) and Sequential Flotant Forward Selection (SFFS) built using the Random Forest (RF) algorithm, for dimensionality reduction of spectral data and predictive modelling of modelling soil organic matter (SOM), clay and carbonates. The reflectance of 100 soil samples, acquired from Sierra de las Nieves (Spain), was measured under laboratory conditions using ASD FieldSpec Pro JR. Four different datasets were obtained after applying two spectral preprocessing methods to raw spectra: raw spectra, Continuum Removal (CR), Multiplicative Scatter Correction (MSC), and a so-called “Global” dataset composed of raw, CR and MSC features. The performance of RF models built with feature selection methods was compared to that of Partial Least Squares Regression (PLSR) and RF (alone).RF models built with SFS and SFFS outperformed PLSR and RF alone models: The best RF models with feature selection had a respective ratio of performance to interquartile distance of 1.93, 0.38 and 2.56. PLSR models had an accuracy of 1.41, 0.29 and 1.81 for SOM, carbonates, and clay, respectively. RF alone had a respective performance of 1.29, 0.29 and 1.81. The application of feature selection wrapper methods reduced the number of features to less than 1% of the starting features. Features were selected across all spectra for SOM and clay, and around 900 nm, 1900 nm, and 2350 nm for carbonates. However, feature selection highlighted features around 1100 nm in SOM modelling, as well as other features around 2200 nm, which is considered a main absorption feature of clay. The application of feature selection with Random Forest was very important in improving modelling accuracy, reducing the redundant features and avoiding the curse of dimensionality or Hughes effect. Thus, this research showed an alternative to dimensionality reduction approaches that have been applied to date to model soil properties with spectroscopy and paves the way for further scientific investigation based on feature selection methods and machine learning.

Immediate traceability of ash based on laser-induced breakdown spectroscopy and machine learning

This article reports on an advanced level physical laboratory experiment designed for college level undergraduate education and for scholars who need specialized training on using and interpreting Laser-Induced Breakdown Spectroscopy. The technical principle, experimental operation and sample preparation of Laser-Induced Breakdown Spectroscopy are introduced in detail. A presentation and discussion of the use of Laser-Induced Breakdown Spectroscopy in the traceability of four common samples is emphasized. Combining Laser-Induced Breakdown Spectroscopy with Machine Learning, two distinct datasets are constructed through the extraction of spectral features. Dimensionality reduction of spectral data is performed using Linear Discriminant Analysis, while the Random Forest model is employed for provenance classification. Finally, the interpretability of the Random Forest model is leveraged to explore the contributions of different spectral elements to provenance tracing. Results demonstrated the system’s effectiveness in not only accurately identifying ash types but also in elucidating the influential chemical components, offering significant implications for material analysis and environmental monitoring. On an educational standpoint, this paper will allow any reader, in particular, undergraduate and graduate students, to gain a better understanding of the theory and practice of Laser-Induced Breakdown Spectroscopy and machine learning.

Development and validation of a clinical-radiomics model for prediction of prostate cancer: a multicenter study.

To develop an early diagnosis model of prostate cancer based on clinical-radiomics to improve the accuracy of imaging diagnosis of prostate cancer. The multicenter study enrolled a total of 449 patients with prostate cancer from December 2017 to January 2022. We retrospectively collected information from 342 patients who underwent prostate biopsy at Minhang Hospital. We extracted T2WI images through 3D-Slice, and used mask tools to mark the prostate area manually. The radiomics features were extracted by Python using the "Pyradiomics" module. Least Absolute Shrinkage and Selection Operator (LASSO) regression was used for data dimensionality reduction and feature selection, and the radiomics score was calculated according to the correlation coefficients. Multivariate logistic regression analysis was used to develop predictive models. We incorporated the radiomics score, PI-RADS, and clinical features, and this was presented as a nomogram. The model was validated using a cohort of 107 patients from the Xuhui Hospital. In total, 110 effective radiomics features were extracted. Finally, 9 features were significantly associated with the diagnosis of prostate cancer, from which we calculated the radiomics score. The predictors contained in the individualized prediction nomogram included age, fPSA/tPSA, PI-RADS, and radiomics score. The clinical-radiomics model showed good discrimination in the validation cohort (C-index = 0.88). This study presents a clinical-radiomics model that incorporates age, fPSA/PSA, PI-RADS, and radiomics score, which can be conveniently used to facilitate individualized prediction of prostate cancer before prostate biopsy.

RadarTCN: Lightweight Online Classification Network for Automotive Radar Targets Based on TCN

Automotive radar is one of the key sensors for intelligent driving. Radar image sequences contain abundant spatial and temporal information, enabling target classification. For existing radar spatiotemporal classifiers, multi-view radar images are usually employed to enhance the information of the target and 3D convolution is employed for spatiotemporal feature extraction. These models consume significant hardware resources and are not applicable to real-time applications. In this paper, RadarTCN, a novel lightweight network, is proposed that achieves high-accuracy online target classification using single-view radar image sequences only. In RadarTCN, 2D convolution and 3D-TCN are employed to extract spatiotemporal features sequentially. To reduce data dimensionality and computational complexity, a multi-layer max pooling down-sampling method is designed in a 2D convolution module. Meanwhile, the 3D-TCN module is improved through residual pruning and causal convolution is introduced for leveraging the performance of online target classification. The experimental results demonstrate that RadarTCN can achieve high-precision online target recognition for both range-angle and range-Doppler map sequences. Compared to the reference models on the CARRADA dataset, RadarTCN exhibits better classification performance, with fewer parameters and lower computational complexity.

A Graph-Based Mathematical Model for More Efficient Dimensionality Reduction of Landmark Data in Geometric Morphometrics

A Graph-Based Mathematical Model for More Efficient Dimensionality Reduction of Landmark Data in Geometric Morphometrics

Quantitative data processing of whole service value chain for power grid internal simulation market

The internal simulation market system can stimulate employee initiative, reduce costs and improve information processing efficiency. However, the complexity of the internal simulation market poses a challenge to computing resources. Efficient data processing techniques are crucial for internal simulation market systems. In this paper, the internal simulation market model and the value chain theory are first put forward. Second, the internal simulation market construction within power grid enterprises is proposed. Then we propose a data mining-based collaborative fusion and processing method for multi-value chain quantitative data in power grid internal simulation markets to conduct data processing, including data fusion, abnormal data elimination and data dimensionality reduction, thereby improving the accuracy of information processing and data sharing. Finally, we validate the superior performance of the proposed method through simulations.

Estimation of the rice aboveground biomass based on the first derivative spectrum and Boruta algorithm

Aboveground biomass (AGB) is regarded as a critical variable in monitoring crop growth and yield. The use of hyperspectral remote sensing has emerged as a viable method for the rapid and precise monitoring of AGB. Due to the extensive dimensionality and volume of hyperspectral data, it is crucial to effectively reduce data dimensionality and select sensitive spectral features to enhance the accuracy of rice AGB estimation models. At present, derivative transform and feature selection algorithms have become important means to solve this problem. However, few studies have systematically evaluated the impact of derivative spectrum combined with feature selection algorithm on rice AGB estimation. To this end, at the Xiaogang Village (Chuzhou City, China) Experimental Base in 2020, this study used an ASD FieldSpec handheld 2 ground spectrometer (Analytical Spectroscopy Devices, Boulder, Colorado, USA) to obtain canopy spectral data at the critical growth stage (tillering, jointing, booting, heading, and maturity stages) of rice, and evaluated the performance of the recursive feature elimination (RFE) and Boruta feature selection algorithm through partial least squares regression (PLSR), principal component regression (PCR), support vector machine (SVM) and ridge regression (RR). Moreover, we analyzed the importance of the optimal derivative spectrum. The findings indicate that (1) as the growth stage progresses, the correlation between rice canopy spectrum and AGB shows a trend from high to low, among which the first derivative spectrum (FD) has the strongest correlation with AGB. (2) The number of feature bands selected by the Boruta algorithm is 19~35, which has a good dimensionality reduction effect. (3) The combination of FD-Boruta-PCR (FB-PCR) demonstrated the best performance in estimating rice AGB, with an increase in R² of approximately 10% ~ 20% and a decrease in RMSE of approximately 0.08% ~ 14%. (4) The best estimation stage is the booting stage, with R2 values between 0.60 and 0.74 and RMSE values between 1288.23 and 1554.82 kg/hm2. This study confirms the accuracy of hyperspectral remote sensing in estimating vegetation biomass and further explores the theoretical foundation and future direction for monitoring rice growth dynamics.

ANDROID APPLICATION FOR AUTOMATIC FILM SELECTION ACCORDING TO USER'S PREFERENCE

The work outlines the development of a movie and TV show recommendation application, integrating with services like IMDb and TMDB, OMDb. It uses Java and Android Architecture Components, with Jsoup for HTML data processing. The app combines content-based and collaborative filtering for personalized recommendations. Collaborative filtering offers diverse suggestions but faces issues like the "cold start" problem, whereas content-based filtering focuses on user-preferred characteristics but may lack variety. To enhance recommendations, Singular Value Decomposition (SVD) can be employed, reducing data dimensionality and revealing hidden relationships, though it has computational limitations.

Performance Comparison of Dimensional Reduction using Principal Component Analysis with Alternating Least Squares in Modified Fuzzy Possibilistic C-Means and Fuzzy Possibilistic C-Means

The clustering method is said to be good if it has resistance to outlier data. One cluster method resistant to outlier data is Fuzzy Possibilistic C-Means (FPCM). FPCM performance on outlier data still has the potential for overlap between cluster members in different clusters, resulting in decreased cluster quality. The Modified Fuzzy Possibilistic C-Means (MFPCM) method is used to modify FPCM in its objective function by inserting updated weight values to increase FPCM performance. In this research, improving the quality of FPCM and MFPCM clusters was carried out by reducing data dimensions through Principal Component Analysis using Alternating Least Squares (PRINCALS) so that members of each cluster do not overlap in the right cluster. The PRINCALS results of the FPCM method have better performance with silhouette values and BSS/TSS ratios of 0.4108 and 60% compared to values without PRINCALS of 0.355 and 43%. The MFPCM method with PRINCALS also performs better, namely 0.4299 and 61%, compared to 0.368 and 42% without PRINCALS. In this study, the performance of MFPCM with PRINCALS or without PRINCALS was better than that of the FPCM method. Overall, PRINCALS can improve the performance of the MFPCM and FPCM methods, resulting in better clusters. PRINCALS in this cluster produce an average silhouette value greater than 0.3 and an average BSS/TSS ratio greater than 50% so that each cluster member is in the right cluster and does not overlap.

Minimax estimation of functional principal components from noisy discretized functional data

AbstractFunctional Principal Component Analysis is a reference method for dimension reduction of curve data. Its theoretical properties are now well understood in the simplified case where the sample curves are fully observed without noise. However, functional data are noisy and necessarily observed on a finite discretization grid. Common practice consists in smoothing the data and then to compute the functional estimates, but the impact of this denoising step on the procedure's statistical performance are rarely considered. Here we prove new convergence rates for functional principal component estimators. We introduce a double asymptotic framework: one corresponding to the sampling size and a second to the size of the grid. We prove that estimates based on projection onto histograms show optimal rates in a minimax sense. Theoretical results are illustrated on simulated data and the method is applied to the visualization of genomic data.

Discrimination of five commercial Guibourtia wood species using terahertz time domain spectroscopy combined with machine learning approaches

Abstract Terahertz waves hold significant potential for applications in wood identification, owing to their good penetration and distinctive fingerprints in wood. This study focuses on wood samples from five different Guibourtia species as the research objects. The terahertz time-domain spectroscopy (THz-TDS) is employed to acquire the spectroscopic signals of the wood samples and to extract their optical parameter data. The THz refractive indices are dimensionally reduced through principal component analysis (PCA), and three machine learning models, namely partial least squares-discriminant analysis (PLS-DA), random forest (RF), and support vector machine (SVM), are employed to classify the wood of five different Guibourtia species. Time delays of the wood samples from five different Guibourtia species are concentrated in the range of 60–62 ps and exhibit different amplitudes in the frequency domain. Refractive indices showed significant variations within the THz band. PCA for dimensionality reduction of terahertz time-domain spectral data significantly improves the recognition rate of machine learning models. Applying PCA to the refractive index data, the RF model achieves a highest recognition rate of 96.9 % and an overall classification accuracy of 98 %. Current results demonstrate that THz-TDS enables rapid, accurate, and non-destructive classification and identification of wood from the Guibourtia species.

Improving with Hybrid Feature Selection in Software Defect Prediction

Software defect prediction (SDP) is used to identify defects in software modules that can be a challenge in software development. This research focuses on the problems that occur in Particle Swarm Optimization (PSO), such as the problem of noisy attributes, high-dimensional data, and premature convergence. So this research focuses on improving PSO performance by using feature selection methods with hybrid techniques to overcome these problems. The feature selection techniques used are Filter and Wrapper. The methods used are Chi-Square (CS), Correlation-Based Feature Selection (CFS), and Forward Selection (FS) because feature selection methods have been proven to overcome data dimensionality problems and eliminate noisy attributes. Feature selection is often used by some researchers to overcome these problems, because these methods have an important function in the process of reducing data dimensions and eliminating uncorrelated attributes that can cause noisy. Naive Bayes algorithm is used to support the process of determining the most optimal class. Performance evaluation will use AUC with an alpha value of 0.050. This hybrid feature selection technique brings significant improvement to PSO performance with a much lower AUC value of 0.00342. Comparison of the significance of AUC with other combinations shows the value of FS PSO of 0.02535, CFS FS PSO of 0.00180, and CS FS PSO of 0.01186. The method in this study contributes to improving PSO in the SDP domain by significantly increasing the AUC value. Therefore, this study highlights the potential of feature selection with hybrid techniques to improve PSO performance in SDP.

Automated Trace Clustering Pipeline Synthesis in Process Mining

Business processes have undergone a significant transformation with the advent of the process-oriented view in organizations. The increasing complexity of business processes and the abundance of event data have driven the development and widespread adoption of process mining techniques. However, the size and noise of event logs pose challenges that require careful analysis. The inclusion of different sets of behaviors within the same business process further complicates data representation, highlighting the continued need for innovative solutions in the evolving field of process mining. Trace clustering is emerging as a solution to improve the interpretation of underlying business processes. Trace clustering offers benefits such as mitigating the impact of outliers, providing valuable insights, reducing data dimensionality, and serving as a preprocessing step in robust pipelines. However, designing an appropriate clustering pipeline can be challenging for non-experts due to the complexity of the process and the number of steps involved. For experts, it can be time-consuming and costly, requiring careful consideration of trade-offs. To address the challenge of pipeline creation, the paper proposes a genetic programming solution for trace clustering pipeline synthesis that optimizes a multi-objective function matching clustering and process quality metrics. The solution is applied to real event logs, and the results demonstrate improved performance in downstream tasks through the identification of sub-logs.

Gauge-Optimal Approximate Learning for Small Data Classification.

Small data learning problems are characterized by a significant discrepancy between the limited number of response variable observations and the large feature space dimension. In this setting, the common learning tools struggle to identify the features important for the classification task from those that bear no relevant information and cannot derive an appropriate learning rule that allows discriminating among different classes. As a potential solution to this problem, here we exploit the idea of reducing and rotating the feature space in a lower-dimensional gauge and propose the gauge-optimal approximate learning (GOAL) algorithm, which provides an analytically tractable joint solution to the dimension reduction, feature segmentation, and classification problems for small data learning problems. We prove that the optimal solution of the GOAL algorithm consists in piecewise-linear functions in the Euclidean space and that it can be approximated through a monotonically convergent algorithm that presents-under the assumption of a discrete segmentation of the feature space-a closed-form solution for each optimization substep and an overall linear iteration cost scaling. The GOAL algorithm has been compared to other state-of-the-art machine learning tools on both synthetic data and challenging real-world applications from climate science and bioinformatics (i.e., prediction of the El Niño Southern Oscillation and inference of epigenetically induced gene-activity networks from limited experimental data). The experimental results show that the proposed algorithm outperforms the reported best competitors for these problems in both learning performance and computational cost.

Advancing forensic-based investigation incorporating slime mould search for gene selection of high-dimensional genetic data

Modern medicine has produced large genetic datasets of high dimensions through advanced gene sequencing technology, and processing these data is of great significance for clinical decision-making. Gene selection (GS) is an important data preprocessing technique that aims to select a subset of feature information to improve performance and reduce data dimensionality. This study proposes an improved wrapper GS method based on forensic-based investigation (FBI). The method introduces the search mechanism of the slime mould algorithm in the FBI to improve the original FBI; the newly proposed algorithm is named SMA_FBI; then GS is performed by converting the continuous optimizer to a binary version of the optimizer through a transfer function. In order to verify the superiority of SMA_FBI, experiments are first executed on the 30-function test set of CEC2017 and compared with 10 original algorithms and 10 state-of-the-art algorithms. The experimental results show that SMA_FBI is better than other algorithms in terms of finding the optimal solution, convergence speed, and robustness. In addition, BSMA_FBI (binary version of SMA_FBI) is compared with 8 binary algorithms on 18 high-dimensional genetic data from the UCI repository. The results indicate that BSMA_FBI is able to obtain high classification accuracy with fewer features selected in GS applications. Therefore, SMA_FBI is considered an optimization tool with great potential for dealing with global optimization problems, and its binary version, BSMA_FBI, can be used for GS tasks.

A novel semi data dimension reduction type weighting scheme of the multi-model ensemble for accurate assessment of twenty-first century drought

A novel semi data dimension reduction type weighting scheme of the multi-model ensemble for accurate assessment of twenty-first century drought

Tracking moisture contents in the pollution layer on a composite insulator surface using hyperspectral imaging technology.

Electrical insulators used in transmission lines and outdoor substations are exposed to severe environmental pollution, which significantly increases the risk of power system failure, especially when the pollution layer is highly humid due to adverse weather conditions. The focus of this paper is to establish an effective method for assessing the moisture content (MC) in pollution layers as it serves as a crucial indicator for evaluating the risk of failure in insulators. Hyperspectral imaging (HSI) technology with a spectral range of 371.08-1037.89 nm was applied to determine significant changes in reflectance spectral characteristics in insulators during dynamic wetting and drying periods. Partial least squares regression (PLSR) models were utilized to evaluate the data presentation enhancement abilities of spectral transformation models and the data dimensionality reduction abilities of characteristic band selection methods. Furthermore, PLSR models were developed to calculate the MC along the pixel dimension to visually retrieve the dynamic wetting and drying processes of the pollution layer. The R-squared and root-mean-square error (RMSE) results in the cross-verification set and prediction set of the RE-RF(70%)-PLSR model with two characteristic bands with a wavelength of 543.28 nm and 848.01 nm were as follows: RCV2 = 0.9824, RMSECV = 0.0367, RP2 = 0.9818, RMSEP = 0.0369, respectively. This research contributes towards the visualization retrieval of the MC and offers an important technique for analyzing flashover evolution, optimizing insulator design, and preparing coating materials for insulators.

Advanced wind turbine blade inspection with hyperspectral imaging and 3D convolutional neural networks for damage detection

In the context of global efforts to mitigate climate change by pursuing sustainable energy sources, wind energy has emerged as a critical contributor. However, the wind energy industry faces substantial challenges in maintaining and preserving the integrity of wind turbine blades. Timely and accurate detection and classification of blade faults, encompassing issues such as cracks, erosion, and ice buildup, are imperative to uphold wind turbines' ongoing efficiency and safety. This study introduces an inventive approach that amalgamates hyperspectral imaging and 3D Convolutional Neural Networks (CNNs) to augment the precision and efficiency of wind turbine blade fault detection and classification. Hyperspectral imaging is harnessed to capture comprehensive spectral information from blade surfaces, facilitating exact fault identification. The process is streamlined through Incremental Principal Component Analysis (IPCA), reducing data dimensions while maintaining integrity. The 3D CNN model demonstrates remarkable performance, achieving high accuracy in detecting all fault categories in full-band hyperspectral images. The model retains high accuracy even with dimensionality reduction to 20 spectral bands. The reduced processing time of the 20-band image enhances the practicality of real-world applications, thereby reducing downtime and maintenance expenditures. This research represents a significant advancement in wind turbine blade inspection, contributing to the sustainability and dependability of wind energy systems and furthering the cause of a cleaner and more sustainable energy future as part of the broader fight against climate change.

Spherical random projection

Abstract We propose a new method for dimension reduction of high-dimensional spherical data based on the nonlinear projection of sphere-valued data to a randomly chosen subsphere. The proposed method, spherical random projection, leads to a probabilistic lower-dimensional mapping of spherical data into a subsphere of the original. In this paper, we investigate some properties of spherical random projection, including expectation preservation and distance concentration, from which we derive an analogue of the Johnson–Lindenstrauss Lemma for spherical random projection. Clustering model selection is discussed as an application of spherical random projection, and numerical experiments are conducted using real and simulated data.