Nonlinear Feature Extraction Research Articles

State of the Art and Future Trends in Data Reduction for High-Performance Computing

Research into data reduction techniques has gained popularity in recent years as storage capacity and performance become a growing concern. This survey paper provides an overview of leveraging points found in high-performance computing (HPC) systems and suitable mechanisms to reduce data volumes. We present the underlying theories and their application throughout the HPC stack and also discuss related hardware acceleration and reduction approaches. After introducing relevant use-cases, an overview of modern lossless and lossy compression algorithms and their respective usage at the application and file system layer is given. In anticipation of their increasing relevance for adaptive and in situ approaches, dimensionality reduction techniques are summarized with a focus on non-linear feature extraction. Adaptive approaches and in situ compression algorithms and frameworks follow. The key stages and new opportunities to deduplication are covered next. An unconventional but promising method is recomputation, which is proposed at last. We conclude the survey with an outlook on future developments.

Deep Learning for Industrial KPI Prediction: When Ensemble Learning Meets Semi-Supervised Data

Soft-sensing techniques are of great significance in industrial processes for monitoring and prediction of key performance indicators. Due to the effectiveness of nonlinear feature extraction and strong expansibility, an autoencoder (AE) and its extensions have been widely developed for industrial applications. Nevertheless, an AE commonly uses the last hidden layer for regression modeling with the output, which seems to be a kind of information waste as the shallow layers are also abstractions of input data. Besides, when there are excessive unlabeled samples, AE-based models are less likely to make full use of them or even degrade the performance. To deal with these issues, a method called ensemble semi-supervised gated stacked AE (ES <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">2</sup> GSAE) is proposed in this article. Gate units are used to develop connections between different layers and the output layer, which also help quantify the contribution of different hidden layers. Moreover, the idea of ensemble learning is combined with semi-supervised learning, in which different unlabeled datasets are used for training different submodels to ensure their diversities. In this way, unlabeled samples can be utilized more efficiently and help enhance the model performance. The effectiveness and superiority are verified in a real industrial process by comparing the proposed method with other typical AE-based models.

Classification with a disordered dopant-atom network in silicon.

Classification is an important task at which both biological and artificial neural networks excel1,2. In machine learning, nonlinear projection into a high-dimensional feature space can make data linearly separable3,4, simplifying the classification of complex features. Such nonlinear projections are computationally expensive in conventional computers. A promising approach is to exploit physical materials systems that perform this nonlinear projection intrinsically, because of their high computational density5, inherent parallelism and energy efficiency6,7. However, existing approaches either rely on the systems' time dynamics, which requires sequential data processing and therefore hinders parallel computation5,6,8, or employ large materials systems that are difficult to scale up7. Here we use a parallel, nanoscale approach inspired by filters in the brain1 and artificial neural networks2 to perform nonlinear classification and feature extraction. We exploit the nonlinearity of hopping conduction9-11 through an electrically tunable network of boron dopant atoms in silicon, reconfiguring the network through artificial evolution to realize different computational functions. We first solve the canonical two-input binary classification problem, realizing all Boolean logic gates12 up to room temperature, demonstrating nonlinear classification with the nanomaterial system. We then evolve our dopant network to realize feature filters2 that can perform four-input binary classification on the Modified National Institute of Standards and Technology handwritten digit database. Implementation of our material-based filters substantially improves the classification accuracy over that of a linear classifier directly applied to the original data13. Our results establish a paradigm of silicon-based electronics for small-footprint and energy-efficient computation14.

Classification of Kidney Cancer Data Using Cost-Sensitive Hybrid Deep Learning Approach

Recently, large-scale bioinformatics and genomic data have been generated using advanced biotechnology methods, thus increasing the importance of analyzing such data. Numerous data mining methods have been developed to process genomic data in the field of bioinformatics. We extracted significant genes for the prognosis prediction of 1157 patients using gene expression data from patients with kidney cancer. We then proposed an end-to-end, cost-sensitive hybrid deep learning (COST-HDL) approach with a cost-sensitive loss function for classification tasks on imbalanced kidney cancer data. Here, we combined the deep symmetric auto encoder; the decoder is symmetric to the encoder in terms of layer structure, with reconstruction loss for non-linear feature extraction and neural network with balanced classification loss for prognosis prediction to address data imbalance problems. Combined clinical data from patients with kidney cancer and gene data were used to determine the optimal classification model and estimate classification accuracy by sample type, primary diagnosis, tumor stage, and vital status as risk factors representing the state of patients. Experimental results showed that the COST-HDL approach was more efficient with gene expression data for kidney cancer prognosis than other conventional machine learning and data mining techniques. These results could be applied to extract features from gene biomarkers for prognosis prediction of kidney cancer and prevention and early diagnosis.

Nonlinear feature extraction and chaos analysis of flow field

A novel signal processing method named adaptive variational mode decomposition with the fractal (AFVMD), which is based on variational mode decomposition and fractal theory, is proposed in this paper for solving a problem that it is easy to misjudge the working conditions of the centrifugal compressor. The measured signal of a compressor is unstable, so a traditional method is used to analyze the nonlinear phenomenon of the stall flutter. Owing to the fact that the robustness of VMD method is strong and its combination with the fractal dimension can accurately describe self-similarity and fractal characteristics of a measured signal, the proposed AFVMD method can not only achieve noise reduction, but also extract nonlinear feature from a complex signal. Taking the dynamic pressure data of the impeller during the instability of a centrifugal compressor as an object to verify the effectiveness and superiority of the proposed AFVMD method, the results are obtained as follows. Firstly, compared with the wavelet noise reduction method, the proposed AFVMD method has both noise reduction and feature extraction functions, and the compressor pressure pulsation spectrum has more significant stall characteristics. Secondly, none of the traditional nonlinear analysis methods can reflect the stall process, so the chaotic phase space attractor is used to visualize the flow field changes. Due to the reasonable choice of the delay time and the embedding dimension, the physical information originally mixed in the signal is separated, so that the attractor phase diagram method has a better process of judging the flow stall than the frequency spectrum method. The results show that the proposed AFVMD method can judge the compressor about to enter into the deep surge earlier. Thirdly, In order to quantify the superiority of the proposed method, if the process of surging and the occurrence of deep wheezing can be predicted in advance, the largest Lyapunov exponent is used as an evaluation index. The above results show that the largest Lyapunov exponent of the proposed AFVMD is smallest for illustrating that the signal has more accurate flow field nonlinear information, which improves the predictability of the signal.

Neural Social Recommendation With User Embedding

Social information is usually jointly utilized with rating information to help the traditional recommendation system providing more personalized services, while how to make full use of social information to build better recommendation models still faces lots of challenges. In this paper, we propose a novel social recommendation model taking advantage of both deep and shallow model, with deep auto-encoders acting as the nonlinear feature extractor and MF-based method being used to depict the user's preferences. Then, motivated by the idea of word2vec, we provide an appealing method that embeds users into latent space and meanwhile preserves the structural information of social networks during the embedding. Also, the embedded latent features of each user are corresponding to the dual roles the user plays in the recommendation. Furthermore, we design a loss function for the holistic training of the model, and our loss function is made up mainly of three parts which embody the effects of different factors on the rating predictions. Specifically, the loss function (i) captures the personal preference from user-item adoption matrix based on matrix factorization, (ii) discriminates the two different social functions of users and further evaluates the effects of interpersonal influence through user embedding and social influence matrix, (iii) and avoids overfitting by imposing a quadratic regularization penalty. As a result, our model can predict the missing ratings with the MF-based method by consuming the latent features of users and items extracted by the deep model. The experiments show that our method outperforms existing methods and performs well on cold start users.

Fast recognition and classification of static and dynamic signs for Persian sign language

Sign Language (SL) is the most effective way for communication between deaf and hearing-impaired people. Since most non-deaf people are not familiar with SL, a vision-based translator/interpreter can be a very useful tool to enhance their communication. This paper presents a recognition system for Persian static and dynamic signs. The system is designed based on proposed modified non-linear kernel-based fast feature extraction methods, consisting of hybrid kernel principal component analysis and hybrid kernel discriminant analysis. For recognition of dynamic signs, the proposed feature extraction method is employed in association with spatio-temporal approach. The proposed methods are examined and compared with several existing feature extraction methods, including linear and non-linear kernel-based methods. The experiments indicate that our feature extraction methods significantly outperform other methods and reduce computational time while they achieve high recognition rates. Our simulations achieved a promising classification accuracy rate of 96.78% on static and 96.99% on dynamic signs, respectively.

Fast recognition and classification of static and dynamic signs for Persian sign language

Sign Language (SL) is the most effective way for communication between deaf and hearing-impaired people. Since most non-deaf people are not familiar with SL, a vision-based translator/interpreter can be a very useful tool to enhance their communication. This paper presents a recognition system for Persian static and dynamic signs. The system is designed based on proposed modified non-linear kernel-based fast feature extraction methods, consisting of hybrid kernel principal component analysis and hybrid kernel discriminant analysis. For recognition of dynamic signs, the proposed feature extraction method is employed in association with spatio-temporal approach. The proposed methods are examined and compared with several existing feature extraction methods, including linear and non-linear kernel-based methods. The experiments indicate that our feature extraction methods significantly outperform other methods and reduce computational time while they achieve high recognition rates. Our simulations achieved a promising classification accuracy rate of 96.78% on static and 96.99% on dynamic signs, respectively.

Diverse Region-Based CNN for Tongue Squamous Cell Carcinoma Classification With Raman Spectroscopy

Border discrimination is very important in the treatment of tongue squamous cell carcinoma (TSCC). This study proposes an ensemble convolutional neural network (CNN) framework based on fiber optic Raman spectroscopy and deep learning techniques to distinguish between TSCC and non-tumor tissue frameworks. First, the data used in the experiments was collected by a fiber optic Raman system. A total of 44 tissues of 22 patients were collected for Raman spectroscopy, with TSCC and adjacent normal tissues each accounting for half. The spectral data range used in the model from a full spectrum of 600-4000 cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> . Then, the ensemble CNN model was used in the experiment. By using two convolution kernels, the model is able to extract nonlinear feature representations from different spectral regions. It has two advantages, on the one hand, it reduces the generation of noise, on the other hand, it obtains a stronger distinguishing ability. Finally, a feature vector is formed by the fusion layer, and is sent to the fully connected layer for the TSCC classification task. The results showed that the sensitivity and specificity of the model were 99.2% and 99.2%, respectively. In addition, comparison with existing methods shows that our method achieves the highest accuracy of TSCC classification. By comparing the different channels, the results show that the spectral range of 1380-2250cm <sup xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">-1</sup> data has the greatest impact on the results. Therefore, Raman spectroscopy combined with the ensemble CNN model has great potential and can provide a useful technique for intraoperative evaluation of the margins of oral tongue squamous cell carcinoma.

Gated Recurrent Unit Networks for Remaining Useful Life Prediction

Remaining useful life prediction is a key procedure for prognostics and health management. However, traditional data-driven methods rely on handcrafted feature selection from the whole range of time series data, which may not obtain the temporal information for complex systems. This study proposes a gated recurrent unit networks based approach to predict remaining useful life. First, time window approach is applied on sample preparation for multiple sensor data. In particular, unsupervised stacked sparse autoencoder is utilized to automatically extract nonlinear features, then the selected features are fed into gated recurrent unit based recurrent neural networks to predict remaining useful life. The effectiveness of the proposed method is demonstrated on the commercial modular aero-propulsion system simulation data from NASA. Experimental results validate that the proposed approach achieves better prediction performance than other methods.

Modeling of Wastewater Treatment Processes Using Dynamic Bayesian Networks Based on Fuzzy PLS

The complicated characteristics of wastewater treatment plants (WWTPs) significantly hinder the monitoring of industrial processes, and thus much attention has been paid to process modeling and prediction. A fuzzy partial least squares-based dynamic Bayesian networks (FPLS-DBN) is proposed to improve the modeling ability in WWTPs. To adapt the nonlinear process data, fuzzy partial least squares (FPLS) is introduced by using a fuzzy system to extract nonlinear features from process data. In addition, a dynamic extension is included by embedding augmented matrices into Bayesian networks to fit the uncertainty and time-varying characteristics. Regarding the quality indices for effluent suspended solid in the WWTP, the root mean square error of the FPLS-DBN model is decreased by 28.63% and 69.47%, respectively, in comparison with that for partial least squares and Bayesian networks. The results demonstrate the superiority of FPLS-DBN in modeling performance for an actual industrial WWTP application.

A Review of Kernel Methods for Feature Extraction in Nonlinear Process Monitoring

Kernel methods are a class of learning machines for the fast recognition of nonlinear patterns in any data set. In this paper, the applications of kernel methods for feature extraction in industrial process monitoring are systematically reviewed. First, we describe the reasons for using kernel methods and contextualize them among other machine learning tools. Second, by reviewing a total of 230 papers, this work has identified 12 major issues surrounding the use of kernel methods for nonlinear feature extraction. Each issue was discussed as to why they are important and how they were addressed through the years by many researchers. We also present a breakdown of the commonly used kernel functions, parameter selection routes, and case studies. Lastly, this review provides an outlook into the future of kernel-based process monitoring, which can hopefully instigate more advanced yet practical solutions in the process industries.

Seizure prediction with cross-higher-order spectral analysis of EEG signals

Epilepsy is a neurological disorder that happens because of the propagation of abnormal signals produced by clusters of neurons in the brain. The majority of those with epileptic seizures can be treated by drug therapies and surgery. However, 25% of the patients with diagnosed epilepsy continue to have seizures. Seizures can cause serious injuries and limit the independence and mobility of an individual. Seizure detection and prediction could lead to a better understanding of seizures and with that help preventing patient injury.This paper discusses extraction and evaluation of nonlinear multivariate features using the cross-bispectral method to help predict epileptic seizure occurrences. These ten statistic features were employed to discriminate pre-ictal from interictal states. Therefore, the features were given to the support vector machine classifier as the input. Outputs were then processed in order to evaluate the sensitivity, false positive rate (FPR) and the prediction time. The proposed method obtained sensitivity of 100% and average FPR of 0.044 per hour by using the “Freiburg epileptic seizure prediction” dataset. This high sensitivity index and low FPR index compared with other studies show the ability of cross-higher-order spectral method to analyze epileptic EEG signals. The proposed method is also fast and easy and may be helpful in other applications of EEG analysis such as sleep stage identification and brain–computer interface.

Relationship between mean transit time of retina and various hemodynamic factors using automatic image analysis technique and high speed fluorescein angiography

AbstractPurposeMean transit time (MTT) is defined as the volume/flow ratio of a capillary bed and reflects tissue perfusion delay. Using high speed fluorescein angiography (FA), image registration algorithm and deconvolution method, MTT of retina was automatically calculated and analyzed with other perfusion factors.MethodsA camera type FA capable of recording an image of 1920 square pixels at 20 frames per second was constructed. Sodium fluorescein (4.0 mL of a 10%) was injected into the antecubital vein to all 6 normal subjects. Automatic image registration was performed to correct saccadic eye movements with the nonlinear diffusion feature extraction (KAZE) method. After selection of the specific major arteries and veins, MTT was calculated by deconvolution method using time series of the fluorescein intensity of each vessel. The relationship between arteriovenous passage time (AVP), arteriovenous transit time (AVT), peak arterial time (PA), peak venous time (PV) and MTT was investigated using correlation analysis.ResultsThe mean age of subjects was 35.2 ± 6.1 years and 24 major arteries and veins were selected. The mean MTT was 1.970 ± 0.898 s. The mean AVP, AVT, PA, and PV were 1.450 ± 0.454, 11.227 ± 1.339, 5.400 ± 1.113, and 7.280 ± 1.368 s. MTT showed a strong correlation with AVP (r = 0.736, p = 0.006), PA (r = 0.637, p = 0.026), and PV (r = 0.769, p = 0.003), but no significant correlation with AVT. AVT was only correlated with PV (r = 0.666, p = 0.018).ConclusionsIn normal subjects, MTT of retina correlated with retinal arterial, venous, and tissue transit times. MTT seems to reflect the perfusion status of all vascular components compared with AVT. A comparative study of patients expected to delay retinal MTT need to be conducted to investigate the clinical significance of MTT.

Supervised kernel discriminant local tangent space alignment for high-resolution range profile-based radar target recognition

We present a modified local tangent space alignment (LTSA) algorithm, called supervised kernel discriminant local tangent space alignment (SKDLTSA), for radar target recognition based on high-resolution range profile (HRRP). SKDLTSA aims to extract intraclass geometric structure embedded in local neighborhoods, as well as to maximize interclass separability characterized by overall distances among different classes. It is formulated with kernel technique to extract nonlinear features, which helps to obtain better performance than its linear counterparts. Extensive experiments on measured HRRP data from three flying airplanes demonstrate that the proposed method can significantly improve the recognition performance. Further results also indicate its robustness to target attitude variations and noise corruption.

Probabilistic just-in-time approach for nonlinear modeling with Bayesian nonlinear feature extraction

In this work, we propose a probabilistic just-in-time (PJIT) modeling methodology with nonlinear feature extraction for estimating quality variables of interest. In literature, deterministic nonlinear feature extraction methods have been employed to deal with high dimensional input data. However, these methods require prespecifying the latent dimensions, which often results in overfitting. To circumvent this issue, we employ the Bayesian Gaussian process latent variable model (BGPLVM) to extract nonlinear latent variables and determine their dimensions automatically. Owing to the probabilistic framework, the proposed approach involves computing the variational distribution of latent variables for the query sample as well as historical samples, and selecting relevant samples based on a distribution measure for building a local Gaussian process model to predict the quality variable. Furthermore, the applicability of the proposed approach to missing data and multi-rate data is discussed. Two case studies are presented to demonstrate the efficacy of the proposed PJIT model.

An integrated method based on refined composite multivariate hierarchical permutation entropy and random forest and its application in rotating machinery

Fault feature extraction of rotating machinery is crucial and challenging due to its nonlinear and nonstationary characteristics. In order to resolve this difficulty, a quality nonlinear fault feature extraction method is required. Hierarchical permutation entropy has been proven to be a promising nonlinear feature extraction method for fault diagnosis of rotating machinery. Compared with multiscale permutation entropy, hierarchical permutation entropy considers the fault information hidden in both high frequency and low frequency components. However, hierarchical permutation entropy still has some shortcomings, such as poor statistical stability for short time series and inability of analyzing multichannel signals. To address such disadvantages, this paper proposes a new entropy method, called refined composite multivariate hierarchical permutation entropy. Refined composite multivariate hierarchical permutation entropy can extract rich fault information hidden in multichannel signals synchronously. Based on refined composite multivariate hierarchical permutation entropy and random forest, a novel fault diagnosis framework is proposed in this paper. The effectiveness of the proposed method is validated using experimental and simulated signals. The results demonstrate that the proposed method outperforms multivariate multiscale fuzzy entropy, refined composite multivariate multiscale fuzzy entropy, multivariate multiscale sample entropy, multivariate multiscale permutation entropy, multivariate hierarchical permutation entropy, and composite multivariate hierarchical permutation entropy in recognizing the different faults of rotating machinery.

Deep Principal Component Analysis Based on Layerwise Feature Extraction and Its Application to Nonlinear Process Monitoring

In order to deeply exploit intrinsic data feature information hidden among the process data, an improved kernel principal component analysis (KPCA) method is proposed, which is referred to as deep principal component analysis (DePCA). Specifically, motivated by the deep learning strategy, we design a hierarchical statistical model structure to extract multilayer data features, including both the linear and nonlinear principal components. To reduce the computation complexity in nonlinear feature extraction, the feature-samples’ selection technique is applied to build the sparse kernel model for DePCA. To integrate the monitoring statistics at each feature layer, Bayesian inference is used to transform the monitoring statistics into fault probabilities, and then, two probability-based DePCA monitoring statistics are constructed by weighting the fault probabilities at all the feature layers. Two case studies involving a simulated nonlinear system and the benchmark Tennessee Eastman process demonstrate the superior fault detection performance of the proposed DePCA method over the traditional KPCA-based methods.

SCAFE: Automated simultaneous clustering and non-linear feature extraction of building energy profiles

Increasing amounts of energy consumption data available to researchers in recent years make it easier to analyze the global energy demands. The analysis is important because buildings consume roughly 40% of global energy. One of the applications of data-driven analysis is identifying buildings that follow similar energy consumption patterns. Despite the number of available unsupervised machine learning algorithms, the user is still required to choose features, typically resorting to hourly data. In this paper, we propose we present SCAFE, an end-to-end architecture for Simultaneous Clustering And non-linear Feature Extraction. The method consists of first transforming the raw energy profiles that are represented as time-series into three channel images, using Gramian Summation, Gramian Difference and Markov Transition Fields. Then, we map the images to the latent space with the use of an autoencoder. Finally, we perform clustering in the latent space, simultaneously obtaining clusters and features of the dataset. We demonstrate SCAFE on the UTexas dataset, showing similar performance to standard k-means clustering while in addition also extracting natural features of the data in the latent space.

Evaluation of Feature Selection and Feature Extraction Techniques on Multi-Temporal Landsat-8 Images for Crop Classification

Recent advancements in the remotely sensed data products and machine learning algorithms are utilized effectively for classification of crops over a considerable large area. This article proposes the use of feature extraction techniques to be employed on the multi-temporal Landsat-8 OLI sensor’s surface reflectances and derived Normalized Difference Indices datasets to classify different crop types. Numerous dimension reduction techniques, viz., feature selection (random forest and PIC measure based), linear (principal component analysis (PCA) and independent component analysis) and nonlinear feature extraction (kernel PCA and Autoencoder), are evaluated to detect most favourable features which should be apt for classification of crops. Subsequently, the detected features are used in a promising nonparametric classifier, support vector machine, for crop classification. It has been found that all the evaluated feature extraction techniques, employed on the multi-temporal datasets, result in better performance compared to feature selection-based approaches. PCA, being a simple and efficient feature extraction algorithm, is well-suited in this classification study and extracted features can classify the crops with an average overall accuracy of 94.32%. Most of the crop types achieve user and producer accuracy of more than 90%. Multi-temporal images prove to be more advantageous compared to the single-date imagery for crop identification.