Entropy-based Feature Extraction Research Articles

Cross Diversity Entropy-Based Feature Extraction for Fault Diagnosis of Rotor System

Cross Diversity Entropy-Based Feature Extraction for Fault Diagnosis of Rotor System

Investigation on the fusion reliability and cluster consistency of multivariable entropy method

Recent researches have shown that the multivariable entropy based feature extraction method can obtain better diagnosis results for fault diagnosis of planetary gearbox. However, the nature properties of multivariable entropy have still not been deeply explored: the reliability of multi-source information fusion and cluster consistency for the same fault signal. These two properties will affect the accuracy of fault diagnosis based on multivariate entropy. This paper aims to reveal the nature properties of multivariate entropy. Firstly, a rigid-flexible coupling dynamic model of a planetary gearbox is conducted to establish a pure test environment. Then the generated vibration signals are used to evaluate the fusion reliability and cluster consistency of multivariable entropy. Additionally, a new multivariable entropy feature extraction method called variational embedding refined composite multiscale diversity entropy (veRCMDE) is proposed. Finally, the simulation and experiment results show that high fusion reliability and high cluster consistency enable multivariate entropy to extract more valuable features, and the proposed veRCMDE performs the best in all experiments.

Entropy and K-Nearest Neighbors-Based Feature Extraction for Bearing Fault Detection

Bearing failures in rotating machines can lead to significant operational challenges, causing up to 45-55% of engine failures and severely impacting performance and productivity. Timely detection of bearing anomalies is crucial to prevent machine failures and associated downtime. Therefore, an approach for early bearing failure detection using entropy-based machine learning is proposed and evaluated while combined with a classifier based on K-Nearest Neighbors (KNN) and Support Vector Machine (SVM). Entropy-based feature extraction should be able to effectively capture the intricate patterns and variations present in the vibration signals, providing a comprehensive representation of the underlying dynamics. The results of the classification carried out by KNN-Entropy have an accuracy value of 98%, while the SVM-Entropy model has an accuracy of 96%. Hence, the Entropy-based feature extraction giving the best accuracy when it is coupled with KNN.

Refined composite multiscale dynamic causality diagram: a novel time-series analysis approach and its application in fault diagnosis of rotating machinery

Entropy-based feature extraction methods have been widely used in the fault diagnosis of rotating machinery, but the entropy-based methods still have the defects of poor noise robustness, weak feature extraction, and low computational efficiency. To solve this problem, this article proposes a fault diagnosis method based on refined composite multiscale dynamic causal diagram (RCMSDCD) and local receptive field extreme learning machine (LRFELM). First, a novel feature extraction method, named dynamic causal diagram (DCD), is proposed to comprehensively quantify static and dynamic complexity. DCD is obtained by combining generalized inverse fractional order entropy with complexity–entropy causal plane. Then, combined with the coarse-graining process, DCD is extended to a multiscale analysis called RCMSDCD to complement the feature description at cross-time scales. Third, RCMSDCD features are input into LRFELM classifier for fault recognition of rotating machinery. The effectiveness of the proposed RCMSDCD-LRFELM method is verified by the Paderborn University bearing test and real wind turbine gear signals. The results show that this method has the highest classification accuracy of 100% with high computational efficiency, good stability, and strong generalization ability.

Multiscale Entropy-Based Feature Extraction for the Detection of Instability Inception in Axial Compressors.

The detection of instability inception is favorable to avoid compressor instability. In this paper, a multiscale entropy-based feature extraction is developed for the detection of the instability inception in axial compressors. Nonlinear and statistical features of the short-time instability inception are extracted by generally combining multiscale entropy and statistical features. First, nonlinear features are extracted by refined composite multiscale entropy to avoid the inaccurate estimation or undefined entropy of multiscale entropy for short time series. Second, the time-domain-based statistical features are chosen to capture more information on instability inception, and the dominant statistical features are determined by random forests implemented with the mean decrease accuracy algorithm at each time scale. The obtained refined composite dominant statistical features are regarded as weighting factors and integrated with the refined composite multiscale entropy to generate a combined feature. Finally, numerical simulation results on two synthetic noise datasets and a compressor instability model dataset are presented to demonstrate the effectiveness, efficiency, and robustness of the combined features under different conditions.

Cloud ensemble learning for fault diagnosis of rolling bearings with stochastic configuration networks

The scarcity of fault samples poses a significant challenge for fault diagnosis of rolling bearings in industrial environment. Conventional fault diagnosis methods struggle to achieve satisfactory results in the few-shot scenarios. Furthermore, common entropy-based feature extraction methods fail to adaptively represent the uncertainty information hidden in vibration signals. To overcome these issues, this article proposes a stochastic configuration network-based cloud ensemble learning (SCN-CEL). Firstly, a cloud feature extraction method is developed to effectively capture fault information from vibration signals while accounting for their inherent uncertainty based on backward cloud generator (BCG) of cloud model (CM), without requiring hyperparameter settings. Subsequently, a cloud oversampling (COS) method is proposed to augment the feature space of limited samples and generate sufficient samples for improving diagnostic accuracy based on bidirectional cloud generator. Finally, we introduce an ensemble model that combines SCNs with multiple constrained COS to comprehensively characterize uncertain fault information and advance the generalization of diagnosis machine. By harnessing the constructive incremental learning of SCN, SCN-CEL guarantees both efficient modeling and accurate prediction for bearing fault diagnosis. Extensive experiments evaluate the effectiveness of each module in SCN-CEL and demonstrate its favorable performance in distinguishing fault categories of rolling bearings in the few-shot scenarios.

An intelligent adverse delivery outcomes prediction model based on the fusion of multiple obstetric clinical data

Adverse delivery outcomes is a major re-productive health problem that affects the physical and mental health of pregnant women. Obviously, obstetric clinical data has periodically time series characteristics. This paper proposed a three stage adverse delivery outcomes prediction model via the fusion of multiple time series clinical data. The first stage is data aggregation, in which the data set is collected from the obstetric clinical data and divided based on time series features. In the second stage, a multi-channel gated cycle unit is used to solve the calculation error caused by irregular sampling of time series data. The hidden layer feature vector is connected with the fully connected layer, reshaped into a new one-dimensional feature, and fused with the non-time series data into a new data set. The third stage is the prediction stage of adverse delivery outcomes. By connecting the multi-channel gated cycle unit with the extreme gradient lift, the data transmitted in the corresponding channel is used in the feature extraction stage, in which the weighted entropy-based feature extraction is adopted. With the help of the extracted features, a hybrid artificial neural network architecture (MGRU-XGB) was developed to predict adverse delivery outcomes. The experimental results showed that the hybrid model had the best prediction performance for adverse delivery outcomes compared with other single models in terms of sensitivity, specificity, AUC and other evaluation indexes.

Comparative Study on Feature Extraction of Marine Background Noise Based on Nonlinear Dynamic Features.

Marine background noise (MBN) is the background noise of the marine environment, which can be used to invert the parameters of the marine environment. However, due to the complexity of the marine environment, it is difficult to extract the features of the MBN. In this paper, we study the feature extraction method of MBN based on nonlinear dynamics features, where the nonlinear dynamical features include two main categories: entropy and Lempel-Ziv complexity (LZC). We have performed single feature and multiple feature comparative experiments on feature extraction based on entropy and LZC, respectively: for entropy-based feature extraction experiments, we compared feature extraction methods based on dispersion entropy (DE), permutation entropy (PE), fuzzy entropy (FE), and sample entropy (SE); for LZC-based feature extraction experiments, we compared feature extraction methods based on LZC, dispersion LZC (DLZC) and permutation LZC (PLZC), and dispersion entropy-based LZC (DELZC). The simulation experiments prove that all kinds of nonlinear dynamics features can effectively detect the change of time series complexity, and the actual experimental results show that regardless of the entropy-based feature extraction method or LZC-based feature extraction method, they both present better feature extraction performance for MBN.

Entropy-Based Feature Extraction Model for Fundus Images with Deep Learning Model

Diabetic retinopathy (DR) is stated as a disease in the eyes that affects the retina blood vessels and causes blindness. The early diagnosis and detection of the DR in patients preserve the patient’s vision. In general, for the diagnosis of eye diseases, retinal fundus images are employed. The advancement in the automatic diagnosis of diseases attained higher significance for rapid advancement in computing technology in the medical field. Besides, for the diagnosis of the diseases, fundus image automatic detection is involved in the recognition of blood vessels evaluated based on the length, branching pattern, and width. However, fundus images have low contrast and it is difficult to evaluate the identification of the disease in blood vessels. As a result, it is necessary to adopt a consistent automated method to extract blood vessels in the fundus images for DR. The conventional automated localization of the macula and optic disk in the retinal fundus images needs to be improved for DR disease diagnosis. But existing methods are not sufficient for the early identification and detection of DR. This paper proposed an entropy distributed matching global and local clustering (EDMGL) for fundus images. The developed EDMGL comprises the different uncertainties for the evaluation of the classes based on local and global entropy. The fundus image local entropy is evaluated based on the spatial likelihood fuzzifier membership function estimation for segmentation. The final proposed algorithm membership function is estimated using the addition of weighted parameters through membership estimation based on the global and local entropy. The classification performance of the proposed EDMGL is evaluated based on the dice coefficient, segmentation accuracy, and partition entropy. The performance of the proposed EDMGL is comparatively examined with the conventional technique. The comparative analysis expressed that the performance of the proposed EDMGL exhibits [Formula: see text]5% improved performance in terms of accuracy, precision, recall, and F1-score.

Compound mechanical fault diagnosis based on CMDE

The fault diagnosis technique is of important for the safety operation of the rotating machinery. In the fault diagnosis framework, the entropy-based method is a promising tool for the feature extraction and signal processing. Among the entropy-based methods, the diversity entropy has arisen increasing attention due to its merits of high consistency, strong robustness, and high calculation efficiency. However, it suffers the defect that the multiscale procedure leads to unstable complexity estimation at higher scales. This induces a poor cluster performance in analyzing the compound mechanical fault signals. To address this issue, this paper presents a novel feature extraction method called composite multiscale diversity entropy (CMDE). The proposed CMDE utilizes the mean complexity value of multiple sliding windows for each scale to enhance the stability, which enables the diversity entropy could dig richer fault information from deeper scales for the compound fault diagnosis of rotating machinery. Then, the stability of CMDE has been evaluated using synthetic gear signals. At last, the proposed CMDE has been applied in the compound mechanical fault diagnosis. The experimental results show that the CMDE achieves the highest diagnosis accuracy compared to the existing entropy-based feature extraction methods.

Generalized Composite Multiscale Diversity Entropy and Its Application for Fault Diagnosis of Rolling Bearing in Automotive Production Line

This paper considers the entropy based feature extraction method for the fault diagnosis of rolling bearings in automobile production line, where the fault information is difficult to identify due to the strong nonlinear and non-stationary characteristics of the fault vibration signals. In our work, a novel entropy based method called generalized composite multiscale diversity entropy (GCMDiEn) is developed. This method can effectively track the inside pattern changes of the time series by the description of cosine similarity between adjacent orbits. Unlike most of the existing entropy based results which concentrate on the static orderliness, we analyze the dynamic complexity characteristics of the arbitrary time series. Moreover, compared with the multiscale diversity entropy, GCMDiEn calculates all coarse-grained time series entropy value with the same scale and extends the first-order moment to second-order moment to mitigate the shortcomings of the short time series instability and losing part of the frequency region information. This paper proposed a new rolling bearing fault diagnosis framework which combined the GCMDiEn with the empirical wavelet transform (EWT), Laplacian score (LS), and particle swarm optimization-based support vector machine (PSO-SVM). Finally, the simulation results show the superiority of the GCMDiEn method over the multiscale diversity entropy method. The proposed framework has a higher fault recognition rate (99.38%) than the existing methods.

Unsupervised Doppler Radar Based Activity Recognition for e-Healthcare

Passive radio frequency (RF) sensing and monitoring of human daily activities in elderly care homes is an emerging topic. Micro-Doppler radars are an appealing solution considering their non-intrusiveness, deep penetration, and high-distance range. Unsupervised activity recognition using Doppler radar data has not received attention, in spite of its importance in case of unlabelled or poorly labelled activities in real scenarios. This study proposes two unsupervised feature extraction methods for the purpose of human activity monitoring using Doppler-streams. These include a local Discrete Cosine Transform (DCT)-based feature extraction method and a local entropy-based feature extraction method. In addition, a novel application of Convolutional Variational Autoencoder (CVAE) feature extraction is employed for the first time for Doppler radar data. The three feature extraction architectures are compared with the previously used Convolutional Autoencoder (CAE) and linear feature extraction based on Principal Component Analysis (PCA) and 2DPCA. Unsupervised clustering is performed using K-Means and K-Medoids. The results show the superiority of DCT-based method, entropy-based method, and CVAE features compared to CAE, PCA, and 2DPCA, with more than 5\%-20\% average accuracy. In regards to computation time, the two proposed methods are noticeably much faster than the existing CVAE. Furthermore, for high-dimensional data visualisation, three manifold learning techniques are considered. The methods are compared for the projection of raw data as well as the encoded CVAE features. All three methods show an improved visualisation ability when applied to the encoded CVAE features.

Hierarchical multiscale permutation entropy-based feature extraction and fuzzy support tensor machine with pinball loss for bearing fault identification

Recently, tensor learning has been applied to intelligent bearing health monitoring successfully. In the past tensor learning based fault identification approaches, the clipped time–frequency images (TFIs) are usually taken as the input tensor data. However, the manual clipping method is usually used to reduce the dimension of TFIs, which may easily lose some critical fault information. Hence, we present hierarchical multiscale permutation entropy (HMPE) as a new feature representation method to avert this drawback from the perspective of statistical measure. In practice, the extracted fault feature (HMPE) may often be corrupted by noise. Hence, fuzzy support tensor machine with pinball loss (Pin-FSTM) as a new tensor-based classifier is developed, which can reduce noise sensitivity and minimize classification errors. Finally, a novel bearing health monitoring strategy based on HMPE and Pin-FSTM is presented, which has been demonstrated to have an excellent identification performance through two experiments. Compared with the state-of-the-art permutation entropy-based fault identification algorithms, the presented scheme is more superior in accuracy and stability. Besides, this research provides a new direction for tensor learning further applying in bearing health monitoring.

Feature Extraction of Ship-Radiated Noise Based on Enhanced Variational Mode Decomposition, Normalized Correlation Coefficient and Permutation Entropy.

Due to the complexity and variability of underwater acoustic channels, ship-radiated noise (SRN) detected using the passive sonar is prone to be distorted. The entropy-based feature extraction method can improve this situation, to some extent. However, it is impractical to directly extract the entropy feature for the detected SRN signals. In addition, the existing conventional methods have a lack of suitable de-noising processing under the presence of marine environmental noise. To this end, this paper proposes a novel feature extraction method based on enhanced variational mode decomposition (EVMD), normalized correlation coefficient (norCC), permutation entropy (PE), and the particle swarm optimization-based support vector machine (PSO-SVM). Firstly, EVMD is utilized to obtain a group of intrinsic mode functions (IMFs) from the SRN signals. The noise-dominant IMFs are then eliminated by a de-noising processing prior to PE calculation. Next, the correlation coefficient between each signal-dominant IMF and the raw signal and PE of each signal-dominant IMF are calculated, respectively. After this, the norCC is used to weigh the corresponding PE and the sum of these weighted PE is considered as the final feature parameter. Finally, the feature vectors are fed into the PSO-SVM multi-class classifier to classify the SRN samples. The experimental results demonstrate that the recognition rate of the proposed methodology is up to 100%, which is much higher than the currently existing methods. Hence, the method proposed in this paper is more suitable for the feature extraction of SRN signals.

A malware variants detection methodology with an opcode-based feature learning method and a fast density-based clustering algorithm

Malware is one of the most terrible and major security threats facing the internet today, which can be defined as any type of malicious code to harm a computer or network. As malware variants can be equipped with sophisticated mechanisms to bypass traditional detection systems, in this paper, we propose a malware variant detection approach that can automatically, rapidly and accurately detect malware variants. In our approach, we present an asynchronous architecture for automated training and detection. Under this architecture, to improve the detection speed while retaining the accuracy, we propose an information entropy-based feature extraction method to extract a few but very useful features and a distance-based weight learning method to weight these features. To further improve the detection speed, we propose our fast density-based clustering algorithm. We evaluate our approach with a number of Windows-based malware instances which belong to six large families, and our experiments demonstrate that our automated malware variant detection method is able to achieve high accuracy with a significant speedup compared with the other state-of-art approaches.

Classification of Hepatitis Viruses from Sequencing Chromatograms Using Multiscale Permutation Entropy and Support Vector Machines

Classifying nucleic acid trace files is an important issue in molecular biology researches. For the purpose of obtaining better classification performance, the question of which features are used and what classifier is implemented to best represent the properties of nucleic acid trace files plays a vital role. In this study, different feature extraction methods based on statistical and entropy theory are utilized to discriminate deoxyribonucleic acid chromatograms, and distinguishing their signals visually is almost impossible. Extracted features are used as the input feature set for the classifiers of Support Vector Machines (SVM) with different kernel functions. The proposed framework is applied to a total number of 200 hepatitis nucleic acid trace files which consist of Hepatitis B Virus (HBV) and Hepatitis C Virus (HCV). While the use of statistical-based feature extraction methods allows representing the properties of hepatitis nucleic acid trace files with descriptive measures such as mean, median and standard deviation, entropy-based feature extraction methods including permutation entropy and multiscale permutation entropy enable quantifying the complexity of these files. The results indicate that using statistical and entropy-based features produces exceptionally high performances in terms of accuracies (reached at nearly 99%) in classifying HBV and HCV.

ECG Based Biometric using Wavelet Packet Decomposition

Biometric technology has been commonly used for authentication. Fingerprint or iris become one of the biometrics that is widely applied. However, this type of biometrics tends to be easily falsified and damaged. So it is misused for manipulating actions and even crime. Therefore a new biometric method is needed to overcome this problem. One potential modality is biometrics based on an electrocardiogram (ECG) signal. This research simulates a one-lead ECG waveform for person authentication. ECG waves were taken from eleven healthy adult volunteers with a length of 60 seconds. ECG waves from each person are segmented into 10 sections so that a total of 110 ECG waves are used for person authentication simulations. All noise of the ECG waves was removed using a bandpass filter to reduce artifacts and high-frequency noise. Wavelet packet decomposition (3 Level) was applied to decompose the signal in several intrinsic parts so that typical wave information can be retrieved. Entropy-based feature extraction applied to all decomposed signals. A total of 14 entropy features have been calculated and used as predictors in the classification process. Validation and performance tests are carried out by cross-validation combined with linear discriminant analysis and support vector machines with five scenarios. The proposed method provides the highest accuracy of 71.8% using discriminant analysis and cubic support vector machine. The best accuracy value was achieved if all entropy features from all wavelet decomposition levels are used as predictors in the classification process. This research is expected to be a reference that ECG has the potential to become a future biometric modality.

A malware variants detection methodology with an opcode-based feature learning method and a fast density-based clustering algorithm

Malware is one of the most terrible and major security threats facing the internet today, which can be defined as any type of malicious code to harm a computer or network. As malware variants can be equipped with sophisticated mechanisms to bypass traditional detection systems, in this paper, we propose a malware variant detection approach that can automatically, rapidly and accurately detect malware variants. In our approach, we present an asynchronous architecture for automated training and detection. Under this architecture, to improve the detection speed while retaining the accuracy, we propose an information entropy-based feature extraction method to extract a few but very useful features and a distance-based weight learning method to weight these features. To further improve the detection speed, we propose our fast density-based clustering algorithm. We evaluate our approach with a number of Windows-based malware instances which belong to six large families, and our experiments demonstrate that our automated malware variant detection method is able to achieve high accuracy with a significant speedup compared with the other state-of-art approaches.

An Improved Refined Composite Multivariate Multiscale Fuzzy Entropy Method for MI-EEG Feature Extraction.

Feature extraction of motor imagery electroencephalogram (MI-EEG) has shown good application prospects in the field of medical health. Also, multivariate entropy-based feature extraction methods have been gradually applied to analyze complex multichannel biomedical signals, such as EEG and electromyography. Compared with traditional multivariate entropies, refined composite multivariate multiscale fuzzy entropy (RCmvMFE) overcomes the defect of unstable entropy values caused by the scale factor increase and is beneficial towards obtaining richer feature information. However, the coarse-grained process of RCmvMFE is mean filtered, which weakens Gaussian noise and is powerless against random impulse noise interference. This yields poor quality feature information and low accuracy classification. In this paper, RCmvMFE is improved (IRCmvMFE) by using composite filters in the coarse-grained procedure to enhance filter performance. Median filters are employed to remove the impulse noise interference from multichannel MI-EEG signals, and these filtered MI-EEGs are further smoothed by the mean filters. The multiscale IRCmvMFEs are calculated for all channels of composite filtered MI-EEGs, forming a feature vector, and a support vector machine is used for pattern classification. Based on two public datasets with different motor imagery tasks, the recognition results of 10 × 10-fold cross-validation achieved 99.43% and 99.86%, respectively, and the statistical analysis of experimental results was completed, showing the effectiveness of IRCmvMFE, as well. The proposed IRCmvMFE-based feature extraction method is superior compared to entropy-based and traditional methods.

Identification of Anesthesia Stages from EEG Signals using Wavelet Entropy and Backpropagation Neural Network

This study focuses on entropy based analysis of EEG signals for extracting features for a neural network based solution for identifying anesthetic levels. The process involves an optimized back propagation neural network with a supervised learning method. We provided the extracted features from EEG signals as training data for the neural network. The target outputs provided are levels of anesthesia stages. Wavelet analysis provides more effective extraction of key features from EEG data than power spectral density analysis using Fourier transform. The key features are used to train the Back Propagation Neural Network (BPNN) for pattern classification network. The final result shows that entropy-based feature extraction is an effective procedure for classifying EEG data.