Sparse Random Projection Research Articles

Temperature rise of high-speed bearing in gearbox of 5 MW wind turbine based on Bayesian-LightGBM and improved PSO-SVM troubleshooting

5MW wind turbine gearbox high-speed bearing temperature rise failure is one of the important factors affecting the stable operation of the wind turbine, accurate prediction and timely diagnosis can effectively improve the efficiency of the wind turbine. In this paper, a combined modelling wind turbine gearbox high-speed bearing temperature rise prediction method based on Bayesian-LightGBM and improved PSO-SVM is proposed with a 5 MW wind turbine as the research object. Firstly, the initial dimensionality reduction of SCADA data is performed by sparse random projection matrix, which reduces the redundant data. Secondly, feature selection is performed on the remaining data using Bayesian-LightGBM to identify 13 key input feature parameters. Then, the hyperparameters of the PSO algorithm are optimised using Bayesian algorithm and further, the optimised PSO algorithm is applied to identify the SVM parameters. Finally, a simulation experiment platform is established based on MATLAB to verify the temperature rise of high-speed bearings in gearboxes of wind turbines by example calculation and comparative analysis. The results show that the model established in this paper is effective, the prediction results are accurate and the performance is stable, and then compared with the algorithms such as PSO-SVM, SVM, BP, etc, the coefficient of determination of the algorithm is greater than 0.994 in both the training set and the test set, and the average decision percentage error is around 1.88% in both.

Determining the time before or after a galaxy merger event

Aims. This work aims to reproduce the time before or after a merger event of merging galaxies from the IllustrisTNG cosmological simulation using machine learning. Methods. Images of merging galaxies were created in the u, g, r, and i bands from IllustrisTNG. The merger times were determined using the time difference between the last simulation snapshot where the merging galaxies were tracked as two galaxies and the first snapshot where the merging galaxies were tracked as a single galaxy. This time was then further refined using simple gravity simulations. These data were then used to train a residual network (ResNet50), a Swin Transformer (Swin), a convolutional neural network (CNN), and an autoencoder (using a single latent neuron) to reproduce the merger time. The full latent space of the autoencoder was also studied to see if it reproduces the merger time better than the other methods. This was done by reducing the latent space dimensions using Isomap, linear discriminant analysis (LDA), neighbourhood components analysis, sparse random projection, truncated singular value decomposition, and uniform manifold approximation and projection. Results. The CNN is the best of all the neural networks. The performance of the autoencoder was close to the CNN, with Swin close behind the autoencoder. ResNet50 performed the worst. The LDA dimensionality reduction performed the best of the six methods used. The exploration of the full latent space produced worse results than the single latent neuron of the autoencoder. For the test data set, we found a median error of 190 Myr, comparable to the time separation between snapshots in IllustrisTNG. Galaxies more than ≈625 Myr before a merger have poorly recovered merger times, as well as galaxies more than ≈125 Myr after a merger event.

Industrial defect detection and location based on greedy particle swarm optimization algorithm

ABSTRACT Particle swarm optimization algorithm is a branch of evolutionary computing and can search for a better solution in a given feature space. This paper introduces the particle swarm optimization algorithm and greedy strategy for the defect detection and location of industrial components and then proposes a greedy particle swarm optimization algorithm. This paper employs sparse random projection to map the vast high-dimensional information to the low-dimensional space while maintaining the relative distance between the data. This employment helps to accelerate the training and prediction speed of the model and remove some unimportant features or noise. This algorithm first adopts particle swarm optimization (PSO) to initialize the cluster centers in an effort to minimize the maximum distance between all unlabeled data points and these centers. The algorithm then utilizes the greedy strategy to select a batch of data points to represent the corresponding features of the normal image, thereby improving the coverage of the model to the data. Experiments have shown that the results of most categories of data sets are close to or better than the current existing methods, especially in defect detection. In terms of defect localization for object categories, our method achieves a pixel-level anomaly localization index (AUROC) of 98.3% on the MVTec AD dataset.

FDEPCA: A Novel Adaptive Nonlinear Feature Extraction Method via Fruit Fly Olfactory Neural Network for IoMT Anomaly Detection.

With the rapid development of 5G communication technology, the data in the Internet of Medical Things (IoMT) application systems exhibits complex characteristics such as large volume, high dimensionality, nonlinearity, and diversity, which significantly affect the efficiency and detection performance of anomaly detection tasks. How to efficiently extract nonlinear features from high-dimensional data in the context of the IoMT while minimizing information distortion in data objects are challenging problems in recent academic research. A novel adaptive nonlinear feature extraction method via fruit fly olfactory neural network (Fly dimension expansion projection and remain main components by PCA, FDEPCA) is proposed, where 1) the data are mean-centered; 2) a binary sparse random projection matrix is used for dimension expansion projection; and 3) PCA is used to extract principal component information. The proposed method overcomes the problems of present nonlinear feature extraction in the face of high-dimensional outliers where the intrinsic geometric structure of the data is severely distorted and computationally expensive. The dataset after nonlinear feature extraction by the FDEPCA algorithm is applied to specific anomaly detection models, using ROC curves and AUC as evaluation metrics for classification performance. Extensive comparison experiments are conducted on eight publicly available datasets, and experimental results show that compared with the popular nonlinear feature extraction algorithms, the FDEPCA algorithm has better classification performance and projection time advantage. When applied to proximity-based, probability-based, and ensemble-based different anomaly detection models respectively, the FDEPCA algorithm exhibits strong applicability in different types of anomaly detection classifiers.

AAD-Net: Advanced end-to-end signal processing system for human emotion detection & recognition using attention-based deep echo state network

Speech signals are the most convenient way of communication between human beings and the eventual method of Human–Computer Interaction (HCI) to exchange emotions and information. Recognizing emotions from speech signals is a challenging task due to the sparse nature of emotional data and features. In this article, we proposed a Deep Echo-State-Network (DeepESN) system for emotion recognition with a dilated convolution neural network and multi-headed attention mechanism. To reduce the model complexity, we incorporate a DeepESN that combines reservoir computing for higher-dimensional mapping. We also used fine-tuned Sparse Random Projection (SRP) to reduce dimensionality and adopted an early fusion strategy to fuse the extracted cues and passed the joint feature vector via a classification layer to recognize emotions. Our proposed model is evaluated on two public speech corpora, EMO-DB and RAVDESS, and tested for subject/speaker-dependent/independent performance. The results show that our proposed system achieves a high recognition rate, 91.14, 85.57 for EMO-DB, and 82.01, 77.02 for RAVDESS, using speaker-dependent and independent experiments, respectively. Our proposed system outperforms the State-of-The-Art (SOTA) while requiring less computational time.

Self-adaptive fuzzy learning ensemble systems with dimensionality compression from data streams

Ensemble learning is a widely used methodology to build powerful predictors from multiple individual weaker ones. However, the vast majority of ensemble learning models are designed for offline application scenarios, the use of evolving fuzzy systems in ensemble learning for online learning from data streams has not been sufficiently explored, yet. In this paper, a novel self-adaptive fuzzy learning ensemble system is introduced for data stream prediction. The proposed ensemble system employs the very sparse random projection technique to compress the consequent parts of the learned fuzzy rules by individual base models to a more compressed form, thereby reducing redundant information and improving computational efficiency. To improve the overall prediction performance, a dynamical base model pruning scheme is introduced to the proposed ensemble system together with a novel inferencing scheme, such that less accurate base models will be removed from the ensemble structure at each learning cycle automatically and only these more accurate ones will be involved in joint decision-making. Numerical examples based on a wide range of benchmark datasets demonstrate the stronger prediction performance of the proposed ensemble system over the state-of-the-art alternatives.

Matrix Factorization-Based Dimensionality Reduction Algorithms─A Comparative Study on Spectroscopic Profiling Data.

Spectroscopic profiling data used in analytical chemistry can be very high-dimensional. Dimensionality reduction (DR) is an effective way to handle the potential "curse of dimensionality" problem. Among the existing DR algorithms, many can be categorized as a matrix factorization (MF) problem, which decomposes the original data matrix X into the product of a low-dimensional matrix W and a dictionary matrix H. First, this paper provides a theoretical reformulation of relevant DR algorithms under a unified MF perspective, including PCA (principal component analysis), NMF (non-negative matrix factorization), LAE (linear autoencoder), RP (random projection), SRP (sparse random projection), VQ (vector quantization), AA (archetypical analysis), and ICA (independent component analysis). From this perspective, an open-sourced toolkit has been developed to integrate all of the above algorithms with a unified API. Second, we made a comparative study on MF-based DR algorithms. In a case study of TOF (time-of-flight) mass spectra, the eight algorithms extracted three components from the original 27,619 features. The results are compared by a set of DR quality metrics, e.g., reconstruction error, pairwise distance/ranking property, computational cost, local and global structure preservations, etc. Finally, based on the case study result, we summarized guidelines for DR algorithm selection. (1) For reconstruction quality, choose ICA. In the case study, ICA, PCA, and NMF have high reconstruction qualities (reconstruction error < 2%), ICA being the best. (2) To keep the pairwise topological structure, choose PCA. PCA best preserves the pairwise distance/ranking property. (3) For edge computing and IoT scenarios, choose RP or SRP if reconstruction is not required and the JL-lemma condition is met. The RP family has the best computational performance in the experiment, almost 10-100 times faster than its peers.

Dimensionality reduction to improve search time and memory footprint in content-retrieval tasks: Application to semiconductor inspection images

Quality control in semiconductors is a crucial step to produce high quality microchips. During the last years, advances in artificial vision have significantly improved image quality control techniques. In the semiconductor industry, automated visual inspection is fundamental to avoid human intervention and keep the pipeline sanitized. Different types of images are collected during this process, feeding image databases that continually grow and cannot be labelled by humans in an exhaustive manner. Advances in image retrieval search methods are fundamental to develop more efficient techniques that meet user requirements.In this work we propose a dimensionality reduction approach on the feature vectors computed by a classifying deep learning model, while keeping a high retrieval performance. To validate this technique, we evaluate four well-known reduction algorithms on a subset of the full database: Principal Component Analysis (PCA), Sparse Random Projection (SRP), Isomap, Locally Linear Embedding (LLE), in combination with three similarity metrics: Euclidian (L2), cosine and inner product. As the number of components of the vectors is reduced, the performance of the image retrieval is measured by recall, time to search, and memory footprint of the database.PCA offers the best results, allowing a significant reduction in search time and memory usage, while SRP becomes an option only when the cosine distance is used. With PCA, we were able to divide the memory footprint by a factor of 16, the search time by 6, while maintaining an average recall of 0.96.

Sparse random projection-based hyperdisk classifier for bevel gearbox fault diagnosis

The fault diagnosis of bevel gearbox is of great significance. At present, the commonly used methods are based on pattern recognition, such as support vector machine, convex hull classifier and hyperdisk classifier. However, the number of elements in the kernel matrix of these kernel function-based classification methods increases squarely with the data size, resulting in intolerable training time. Based on this, a sparse random projection-based hyperdisk classifier model is proposed. The proposed method has the following novelties: First, based on sparse random projection and the geometrical characteristics of the hyperdisk model, a method is designed to efficiently screen out the core samples, and these samples are given different weights in this process. Second, the proposed method introduces slack variables and the dynamic penalty parameter to obtain a hyperdisk model with more reasonable boundary. Last, a strategy is developed to minimize the adverse effects of imbalanced training data. The effectiveness and applicability of the proposed method are verified on bevel gearbox fault data. The experimental results show that compared with other classifiers, the proposed method can greatly reduce the training time while guaranteeing a high classification accuracy. What’s more, it has better performance and efficiency in fault diagnosis with imbalanced training data.

Bidirectional parallel echo state network for speech emotion recognition.

Speech is an effective way for communicating and exchanging complex information between humans. Speech signal has involved a great attention in human-computer interaction. Therefore, emotion recognition from speech has become a hot research topic in the field of interacting machines with humans. In this paper, we proposed a novel speech emotion recognition system by adopting multivariate time series handcrafted feature representation from speech signals. Bidirectional echo state network with two parallel reservoir layers has been applied to capture additional independent information. The parallel reservoirs produce multiple representations for each direction from the bidirectional data with two stages of concatenation. The sparse random projection approach has been adopted to reduce the high-dimensional sparse output for each direction separately from both reservoirs. Random over-sampling and random under-sampling methods are used to overcome the imbalanced nature of the used speech emotion datasets. The performance of the proposed parallel ESN model is evaluated from the speaker-independent experiments on EMO-DB, SAVEE, RAVDESS, and FAU Aibo datasets. The results show that the proposed SER model is superior to the single reservoir and the state-of-the-art studies.

RESERVOIR COMPUTING WITH TRUNCATED NORMAL DISTRIBUTION FOR SPEECH EMOTION RECOGNITION

Speech is an effective, quick, and important way for communicating and exchanging complex information between humans. Emotions have always been a part of normal human conversation which makes the speech more attractive. Because of this major role of both speech and emotion, many researchers are inspired by studying Speech Emotion Recognition (SER) which still has plenty of challenges. In this study, we proposed a novel reservoir computing approach with the initialization of random connection weights for the input weight by the truncated normal distribution. Furthermore, Population-Based Training (PBT) is adopted to optimize the hyperparameters of the whole Echo State Network (ESN) model which have a significant impact on the model performance. The proposed model has adopted bidirectional reservoir input to increase the memorization capability, and Sparse Random Projection (SRP) was applied for dimensional reduction as a simple, unsupervised, and low complexity approach. The speaker-independent strategy was employed on EMODB and SAVEE datasets as an acted speech emotion dataset and Aibo as a non-acted dataset. The model achieved 84.8%, 65.95%, and 45.99% unweighted average recalls on the EMODB, SAVEE, and Aibo datasets respectively. The results show that the proposed model outperforms the recent state-of-the-art studies with a cheaper computational cost.

Analog Circuit Soft Fault Diagnosis Based on Sparse Random Projections and K-Nearest Neighbor

Analog circuit fault diagnosis is a key problem in theory of circuit networks and has been investigated by many researchers in recent years. An approach based on sparse random projections (SRPs) and K-nearest neighbor (KNN) to the realization of analog circuit soft fault diagnosis has been presented in this paper. The proposed method uses the wavelet packet energy spectrum and sparse random projections to preprocess the time response for feature extraction. Then, the variables of the fault features are constructed, which are used to form the observation sequences of K-nearest neighbor classifier. K-nearest neighbor classifier is used to accomplish the fault diagnosis of analog circuit. In this paper, four-opamp biquad high-pass filter has been used as simulation example to verify the effectiveness of the proposed method. The simulations show that the proposed method offers higher correct fault location rate in analog circuit soft fault diagnosis application as compared with the other methods.

Jointly evolving and compressing fuzzy system for feature reduction and classification

Evolving fuzzy systems (EFSs) are a type of adaptive fuzzy rule-based systems which can self-adapt both their structures and parameters simultaneously. However, the existing EFSs suffer from two drawbacks: 1) classical EFSs usually use all input features to model systems, resulting in lengthy fuzzy rules; 2) some redundant information in fuzzy rules may hinder high generalization. To address these two issues, a promising method is proposed in this paper by combining very sparse random projection (VSRP) with a class of EFSs based-on data clouds, called VSRP-AnYa-EFS. The proposed method introduces: 1) a random sparse-Bernoulli (RSB) matrix based-on VSRP is utilized to compress the lengthy antecedent part into a tighter form, triggering a feature-reduction mechanism. By employing VSRP in RSB matrix, some redundant information in fuzzy rules can be filtered; 2) Local learning is used for consequent parameter optimization to suit decoupled behavior of rules after redundant information between rules is deleted. By adopting VSRP and local learning, the proposed VSRP-AnYa-EFS owns a compact structure and fast learning speed. Numerical examples presented in this paper demonstrate that the proposed method can significantly reduce training time from hours to minutes while the accuracy can be improved up to 5%.

Vehicle Image Recognition Using Deep Convolution Neural Network and Compressed Dictionary Learning

In this paper, a vehicle recognition algorithm based on deep convolutional neural network and compression dictionary is proposed. Firstly, the network structure of fine vehicle recognition based on convolutional neural network is introduced. Then, a vehicle recognition system based on multi-scale pyramid convolutional neural network is constructed. The contribution of different networks to the recognition results is adjusted by the adaptive fusion method that adjusts the network according to the recognition accuracy of a single network. The proportion of output in the network output of the entire multiscale network. Then, the compressed dictionary learning and the data dimension reduction are carried out using the effective block structure method combined with very sparse random projection matrix, which solves the computational complexity caused by high-dimensional features and shortens the dictionary learning time. Finally, the sparse representation classification method is used to realize vehicle type recognition. The experimental results show that the detection effect of the proposed algorithm is stable in sunny, cloudy and rainy weather, and it has strong adaptability to typical application scenarios such as occlusion and blurring, with an average recognition rate of more than 95%.

Compressed Imaging Reconstruction with Sparse Random Projection

As the Internet of Things thrives, monitors and cameras produce tons of image data every day. To efficiently process these images, many compressed imaging frameworks are proposed. A compressed imaging framework comprises two parts, image signal measurement and reconstruction. Although a plethora of measurement devices have been designed, the development of the reconstruction is relatively lagging behind. Nowadays, most of existing reconstruction algorithms in compressed imaging are optimization problem solvers based on specific priors. The computation burdens of these optimization algorithms are enormous and the solutions are usually local optimums. Meanwhile, it is inconvenient to deploy these algorithms on cloud, which hinders the popularization of compressed imaging. In this article, we dive deep into the random projection to build reconstruction algorithms for compressed imaging. We first fully utilize the information in the measurement procedure and propose a combinatorial sparse random projection (SRP) reconstruction algorithm. Then, we generalize the SRP to a novel distributed algorithm called Cloud-SRP (CSRP), which enables efficient reconstruction on cloud. Moreover, we explore the combination of SRP with conventional optimization reconstruction algorithms and propose the Iterative-SRP (ISRP), which converges to a guaranteed fixed point. With minor modifications on the naive optimization algorithms, the ISRP yields better reconstructions. Experiments on real ghost imaging reconstruction reveal that our algorithms are effective. And simulation experiments show their advantages over the classical algorithms.

Speech Emotion Recognition by Late Fusion for Bidirectional Reservoir Computing With Random Projection

Many researchers are inspired by studying Speech Emotion Recognition (SER) because it is considered as a key effort in Human-Computer Interaction (HCI). The main focus of this work is to design a model for emotion recognition from speech, which has plenty of challenges within it. Due to the time series and sparse nature of emotion in speech, we have adopted a multivariate time series feature representation of the input data. The work has also adopted the Echo State Network (ESN) which includes reservoir computing as a special case of the Recurrent Neural Network (RNN) to avoid model complexity because of its untrained and sparse nature when mapping the features into a higher dimensional space. Additionally, we applied dimensionality reduction since it offers significant computational advantages by using Sparse Random Projection (SRP). Late fusion of bidirectionality input has been applied to capture additional information independently of the input data. The experiments for speaker-independent and/or speaker-dependent were performed on four common speech emotion datasets which are Emo-DB, SAVEE, RAVDESS, and FAU Aibo Emotion Corpus. The results show that the designed model outperforms the state-of-the-art with a cheaper computation cost.

Study of Spatial–Spectral Feature Extraction Frameworks With 3-D Convolutional Neural Network for Robust Hyperspectral Imagery Classification

Advances in hyperspectral remote sensing have instigated multitude of applications for better understanding of our planet through remote data acquisition and observation of natural phenomena such as weather monitoring and prediction to include tornado, wild fires, global warming, etc. For this, data analysis methods that exploit the rich spectral and spatial information in hyperspectral data are often employed to gain insights about the natural phenomenon. This work presents a new deep learning based hyperspectral data analysis framework, which efficiently utilizes both spatial and spectral information present in the data to achieve superior classification performance. Gabor filtering is used for spatial feature extraction in conjunction with sparse random projections for spectral feature extraction and dimensionality reduction. Finally, supervised classification using a 3-D convolutional neural network was employed to perform a volumetric hyperspectral data analysis. Experimental results reveal that the proposed spatial-spectral hyperspectral data analysis frameworks outperform the conventional 2-D convolution neural network-based spectral-spatial feature extraction techniques.

A Random Walk-Based Energy-Aware Compressive Data Collection for Wireless Sensor Networks

The energy efficiency for data collection is one of the most important research topics in wireless sensor networks (WSNs). As a popular data collection scheme, the compressive sensing- (CS-) based data collection schemes own many advantages from the perspectives of energy efficiency and load balance. Compared to the dense sensing matrices, applications of the sparse random matrices are able to further improve the performance of CS-based data collection schemes. In this paper, we proposed a compressive data collection scheme based on random walks, which exploits the compressibility of data vectors in the network. Each measurement was collected along a random walk that is modeled as a Markov chain. The Minimum Expected Cost Data Collection (MECDC) scheme was proposed to iteratively find the optimal transition probability of the Markov chain such that the expected cost of a random walk could be minimized. In the MECDC scheme, a nonuniform sparse random matrix, which is equivalent to the optimal transition probability matrix, was adopted to accurately recover the original data vector by using the nonuniform sparse random projection (NSRP) estimator. Simulation results showed that the proposed scheme was able to reduce the energy consumption and balance the network load.

Binarization Methods for Motor-Imagery Brain–Computer Interface Classification

Successful motor-imagery brain-computer interface (MI-BCI) algorithms either extract a large number of handcrafted features and train a classifier, or combine feature extraction and classification within deep convolutional neural networks (CNNs). Both approaches typically result in a set of real-valued weights, that pose challenges when targeting real-time execution on tightly resource-constrained devices. We propose methods for each of these approaches that allow transforming real-valued weights to binary numbers for efficient inference. Our first method, based on sparse bipolar random projection, projects a large number of real-valued Riemannian covariance features to a binary space, where a linear SVM classifier can be learned with binary weights too. By tuning the dimension of the binary embedding, we achieve almost the same accuracy in 4-class MI (≤1.27% lower) compared to models with float16 weights, yet delivering a more compact model with simpler operations to execute. Second, we propose to use memory-augmented neural networks (MANNs) for MI-BCI such that the augmented memory is binarized. Our method replaces the fully connected layer of CNNs with a binary augmented memory using bipolar random projection, or learned projection. Our experimental results on EEGNet, an already compact CNN for MI-BCI, show that it can be compressed by $1.28\times $ at iso-accuracy using the random projection. On the other hand, using the learned projection provides 3.89% higher accuracy but increases the memory size by $28.10\times $ .

SRP‐AKAZE: an improved accelerated KAZE algorithm based on sparse random projection

The AKAZE algorithm is a typical image registration algorithm that has the advantage of high computational efficiency based on non‐linear diffusion. However, it is weaker than the scale‐invariant feature transformation (SIFT) algorithm in terms of robustness and stability. We propose a new and improved version of the AKAZE algorithm by using the SIFT descriptor based on sparse random projection (SRP). The proposed method not only retains the advantage of high efficiency of the AKAZE algorithm in feature detection but also has the stability of the SIFT descriptor. Moreover, the computational complexity due to the high dimension of the SIFT descriptor, which limits the speed of feature matching, is drastically reduced by the SRP strategy. Experiments on several benchmark image datasets demonstrate that the proposed algorithm can significantly improve the stability of the AKAZE algorithm, and the results suggest the better matching performance and robustness of the feature descriptor.