Softmax Regression Classifier Research Articles

Conditional generative adversarial network based data augmentation for fault diagnosis of diesel engines applied with infrared thermography and deep convolutional neural network

This paper tries to introduce a new intelligent method for the early fault diagnosis of diesel engines. Firstly, infrared thermography (IRT) is introduced into diesel engine condition monitoring, then infrared images of diesel engines in four health states, such as normal condition, single-cylinder misfire, multi-cylinder misfire and air filter blockage, are collected and the region of interest (ROI) of infrared images are extracted. Next, conditional generative adversarial network (CGAN) is deployed to perform data augmentation on infrared image datasets. Then, deep convolutional neural network (DCNN) and Softmax regression (SR) classifier are used for automatically extracting infrared image fault features and pattern recognition, respectively. Finally, a comparison with three deep learning (DL) models is performed. The validation results show that the data augmentation method proposed in the paper can significantly improve the early fault diagnosis accuracy, and DCNN has the best fault diagnosis effect and resistance to temperature fluctuation interference among the four DL models.

Data-Driven Fault Detection and Classification for MTDC Systems by Integrating HCTSA and Softmax Regression

The requirement of fast fault isolation poses a great challenge to the safe operation of multi-terminal direct current (MTDC) systems. In order to make a better tradeoff between the speed and reliability of the protection scheme, it is imperative to mine more valuable information from fault transient signals. This paper puts forward a data-driven framework capable of digging out and synthesizing multi-dimensional features to achieve fast and reliable DC fault detection and classification in MTDC systems. Highly comparative time-series analysis (HCTSA) is first adopted to extract extensive features with clear physical interpretations from fault current waveforms, and a few features valuable to fault identification are then selected utilizing the greedy forward search. Based on the reduced features, a softmax regression classifier (SRC) is further proposed to calculate the probability of each fault category with a relatively minor on-line computational burden. Numerical simulations carried out in PSCAD/EMTDC have demonstrated the proposed approach is effective under different fault conditions, robust against noise corruptions as well as abnormal samplings, and replicable in various DC grids. In addition, comprehensive comparison studies with conventional derivative-based protection methods and some typical artificial intelligence based (AI-based) methods have been conducted. It is verified that the proposed method has the advantages of higher fault identification accuracy over conventional protections and shallow structure AI-based methods, better interpretability as well as lower on-line computing complexity over the deep architecture AI-based approaches.

Research on hyper-spectral remote sensing image classification by applying stacked de-noising auto-encoders neural network

Hyper-spectral image can provide precise information on land surface targets identification and classification thanks to its advanced feature on spectral resolution. However, due to its complicated hyper-dimension data structure, greater challenge is put on the conventional image classification methods for hyper-spectral images. To fill this technical knowledge gap, we introduce a deep learning-based feature extraction method for hyper-spectral data classification. Firstly, we used a Stacked De-noising Auto-encoders(SDAE) to extract the in-depth features of hyper-spectral image data: a large amount of unlabeled data was pre-trained to extract the depth characteristics of pixels. We added random noise into the input layer of the network to form a de-noising auto-encoder machine and added treated inputs to reconstruct original data. An L-BFGS (Limited-memory quasi-Newton code) was used to optimize the loss function. In the top layer, deep neural network was fine-tuned by a Softmax regression classifier. All these improvements directed towards the model to obtain the image element abstraction and robust expression in the classification task of the hyper-spectral images. We tested the model performance on a Hyspex imaging spectrometer image and found that the newly introduced model framework outperforms other traditional methods including principal component analysis (PCA), and support vector machine (SVM) classifier, combined PCA-SVM classifier, and Minimum Noise Fraction Rotation (MNF)-SVM classifier.

RP-NBSR: A Novel Network Attack Detection Model Based on Machine Learning

The rapid progress of the Internet has exposed networks to an increased number of threats. Intrusion detection technology can effectively protect network security against malicious attacks. In this paper, we propose a ReliefF-P-Naive Bayes and softmax regression (RP-NBSR) model based on machine learning for network attack detection to improve the false detection rate and F1 score of unknown intrusion behavior. In the proposed model, the Pearson correlation coefficient is introduced to compensate for deficiencies in correlation analysis between features by the ReliefF feature selection algorithm, and a ReliefF-Pearson correlation coefficient (ReliefF-P) algorithm is proposed. Then, the Relief-P algorithm is used to preprocess the UNSW-NB15 dataset to remove irrelevant features and obtain a new feature subset. Finally, naïve Bayes and softmax regression (NBSR) classifier is constructed by cascading the naïve Bayes classifier and softmax regression classifier, and an attack detection model based on RP-NBSR is established. The experimental results on the UNSW-NB15 dataset show that the attack detection model based on RP-NBSR has a lower false detection rate and higher F1 score than other detection models.

Unsupervised domain adaptation via enhanced transfer joint matching for bearing fault diagnosis

Recently, domain adaptation (DA) algorithms have been extensively employed in many fault diagnosis applications. Most prior researches perform well under a general assumption: the distributions of samples are balanced. Nevertheless, the unbalanced distributions of samples are common in practical applications which may cause the performances of these researches drop dramatically. To overcome this deficiency, an enhanced transfer joint matching (TJM) approach is proposed in this paper. Two main contributions are concluded as follows. (1) To our knowledge, it is a pioneering work to apply the maximum variance discrepancy (MVD) for combining with the maximum mean discrepancy (MMD) for the feature matching. (2) The row ℓ2-norm is applied for different domains to improve the generalization ability of the proposed model. In addition, z-score normalization is adopted for the softmax regression classifier. Comprehensive experimental results validate that the enhanced TJM can significantly outperform competitive approached for cross-domain bearing defect diagnosis.

Deep Belief Network for Feature Extraction of Urban Artificial Targets

Reducing the dimension of the hyperspectral image data can directly reduce the redundancy of the data, thus improving the accuracy of hyperspectral image classification. In this paper, the deep belief network algorithm in the theory of deep learning is introduced to extract the in-depth features of the imaging spectral image data. Firstly, the original data is mapped to feature space by unsupervised learning methods through the Restricted Boltzmann Machine (RBM). Then, a deep belief network will be formed by superimposed multiple Restricted Boltzmann Machines and training the model parameters by using the greedy algorithm layer by layer. At the same time, as the objective of data dimensionality reduction is achieved, the underground feature construction of the original data will be formed. The final step is to connect the depth features of the output to the Softmax regression classifier to complete the fine-tuning (FT) of the model and the final classification. Experiments using imaging spectral data showing the in-depth features extracted by the profound belief network algorithm have better robustness and separability. It can significantly improve the classification accuracy and has a good application prospect in hyperspectral image information extraction.

Reciprocating compressor fault diagnosis using an optimized convolutional deep belief network

This article proposes an optimized convolutional deep belief network for fault diagnosis of reciprocating compressors. Sparse filtering is first used to compress raw signal into compact time series by refining the most representative information and to reduce the computational burden. Then, the proposed convolutional deep belief network is adopted to learn the unsupervised features of the compressed signal without the need of feature extraction by human effort. To improve the generalization ability of the network, an optimized probabilistic pooling out is proposed in this article to replace the standard one in the pooling layer of the convolutional deep belief network. Finally, the unsupervised features calculated by the optimized convolutional deep belief network are fed as the input of the softmax regression classifier for fault identification. Four types of vibration signals reflecting different operating conditions are collected from the industry to validate the effectiveness of the proposed method. The obtained results demonstrate that the proposed convolutional deep belief network method can achieve a higher classification accuracy rate of up to 91% for fault diagnosis than the traditional methods and accomplish the fault diagnosis of reciprocating compressor effectively.

Unsupervised Mechanical Fault Feature Learning Based on Consistency Inference-Constrained Sparse Filtering

In machinery fault diagnosis, a large amount of monitoring data is often unlabeled, while the number of labeled data is limited. Therefore, learning effective features from massive unlabeled data is a challenging issue for machinery fault diagnosis. In this paper, a simple unsupervised feature learning method, consistency inference-constrained sparse filtering (CICSF), is proposed to learn mechanical fault features with enhanced clustering performance for fault diagnosis. Firstly, inspired by the data augmentation strategy, consistency inference of latent representations for time series (CILRTS) is derived, which infers that training data instances segmented from the same time series should possess consistent latent feature representations. Then, CILRTS is integrated into sparse filtering (SF) as an additional constraint in the latent feature space. The developed CICSF method can optimize the inter-class sparsity and intra-class similarity of the feature distribution simultaneously. Thus, it can learn more effective features from massive unlabeled data. Finally, based on CICSF, a semi-supervised machinery fault diagnosis method is developed. After unsupervised feature learning by CICSF, a softmax regression classifier is trained with limited labeled data to realize machinery fault diagnosis. Experimental results on bearing and gearbox datasets verify the effectiveness of the proposed method. Moreover, comparisons with standard SF and several auto-encoder (AE) variants validate its superiority in unsupervised feature learning and fault diagnosis using limited labeled data.

A smart data-driven rapid method to recognize the strawberry maturity

In recent years, there have been many studies on the recognition of strawberry maturity, but there are still problems such as low recognition accuracy and expensive experimental instruments. These factors make their methods difficult for farmers to use. To solve these problems, we developed a fast, non-destructive, accurate and convenient method for strawberry maturity identification using smartphones. In this paper, strawberry maturity is divided into three levels: mature, nearly-mature and immature. Considering the actual strawberry harvest process and postharvest handling, we focus on the differentiation between the mature and the nearly-mature ones to help farmers reduce possible damage in transit and improve profitability. We obtained the images of strawberries with different maturities at 535 nm and 670 nm wavelengths through a smartphone and got absorbance data by image processing based on the region of interest. The absorbance data were used to establish three maturity recognition models—i.e., multivariate linear, multivariate nonlinear and SoftMax regression classifier. The results showed that the multivariate nonlinear model had the highest identification accuracy (which is over 94%) in the greenhouse. Therefore, this method has considerable potential as a means for rapid recognition of strawberry maturity.

Rotating machinery fault diagnosis based on convolutional neural network and infrared thermal imaging

Rotating machinery is widely applied in industrial applications. Fault diagnosis of rotating machinery is vital in manufacturing system, which can prevent catastrophic failure and reduce financial losses. Recently, Deep Learning (DL)-based fault diagnosis method becomes a hot topic. Convolutional Neural Network (CNN) is an effective DL method to extract the features of raw data automatically. This paper develops a fault diagnosis method using CNN for InfRared Thermal (IRT) image. First, IRT technique is utilized to capture the IRT images of rotating machinery. Second, the CNN is applied to extract fault features from the IRT images. In the end, the obtained features are fed into the Softmax Regression (SR) classifier for fault pattern identification. The effectiveness of the proposed method is validated using two different experimental data. Results show that the proposed method has a superior performance in identification various faults on rotor and bearings comparing with other deep learning models and traditional vibration-based method.

Superpixel segmentation and machine learning classification algorithm for cloud detection in remote‐sensing images

Cloud detection is a fundamental yet challenging topic in remote-sensing image processing. The authors propose a method for multi-dimensional feature extraction and superpixel segmentation, and use a voting-based clustering ensemble to capture the whole target shape. In order to further identify clouds, snow-covered lands, and bright buildings on remote-sensing images, they first implement an Ostu threshold to get high grey-level sub-regions, and then extract the descriptors of these sub-regions and put them into the softmax regression classifier. Regarding these methods, the authors conduct experiments using GF-1 remote-sensing images. The results demonstrate the effectiveness and excellency of their proposed method.

Robust Deep Softmax Regression Against Label Noise for Unsupervised Domain Adaptation

Domain adaptation aims to generalize the classification model from a source domain to a different but related target domain. Recent studies have revealed the benefit of deep convolutional features trained on a large dataset (e.g. ImageNet) in alleviating domain discrepancy. However, literatures show that the transferability of features decreases as (i) the difference between the source and target domains increases, or (ii) the layers are toward the top layers. Therefore, even with deep features, domain adaptation remains necessary. In this paper, we propose a novel unsupervised domain adaptation (UDA) model for deep neural networks, which is learned with the labeled source samples and the unlabeled target ones simultaneously. For target samples without labels, pseudo labels are assigned to them according to their maximum classification scores during training of the UDA model. However, due to the domain discrepancy, label noise generally is inevitable, which degrades the performance of the domain adaptation model. Thus, to effectively utilize the target samples, three specific robust deep softmax regression (RDSR) functions are performed for them with high, medium and low classification confidence respectively. Extensive experiments show that our method yields the state-of-the-art results, demonstrating the effectiveness of the robust deep softmax regression classifier in UDA.

Long Short-Term Memory Networks Based Automatic Feature Extraction for Photovoltaic Array Fault Diagnosis

Photovoltaic (PV) array fault diagnosis is important because it helps reduce energy and revenue losses to PV system operators. It also reduces fire hazards and electric shocks caused by PV array faults. As a result, many machine-learning-based fault diagnosis techniques have been proposed in recent times. Although the fault diagnosis accuracies associated with these techniques have been impressive, most machine learning algorithms rely on manual feature extraction, which is time consuming, expensive, and diagnostic expertise exacting. To address the problem of manual feature extraction, this paper proposes a new PV array fault diagnosis technique capable of automatically extracting features from raw data for PV array fault classification. The proposed technique utilizes long short-term memory networks, which is a deep learning algorithm, for feature extraction. The extracted features feed into a softmax regression classifier for fault diagnosis. The proposed technique exhibits high fault diagnosis accuracies on both noisy and noiseless data. In addition, the results of the proposed technique compare favorably with those of other techniques. It can, therefore, be inferred from the results that the proposed fault diagnosis technique offers an effective approach to automatically extract useful features from raw data and thus remove the need for the manual feature extraction.

A Text Sentiment Classification Modeling Method Based on Coordinated CNN‐LSTM‐Attention Model

The major challenge that text sentiment classification modeling faces is how to capture the intrinsic semantic, emotional dependence information and the key part of the emotional expression of text. To solve this problem, we proposed a Coordinated CNN-LSTM-Attention(CCLA) model. We learned the vector representations of sentence with CCLA unit. Semantic and emotional information of sentences and their relations are adaptively encoded to vector representations of document. We used softmax regression classifier to identify the sentiment tendencies in the text. Compared with other methods, the CCLA model can well capture the local and long distance semantic and emotional information. Experimental results demonstrated the effectiveness of CCLA model. It shows superior performances over several state-of-the-art baseline methods.

Using multi-scale and hierarchical deep convolutional features for 3D semantic classification of TLS point clouds

ABSTRACTPoint cloud classification, which provides meaningful semantic labels to the points in a point cloud, is essential for generating three-dimensional (3D) models. Its automation, however, remains challenging due to varying point densities and irregular point distributions. Adapting existing deep-learning approaches for two-dimensional (2D) image classification to point cloud classification is inefficient and results in the loss of information valuable for point cloud classification. In this article, a new approach that classifies point cloud directly in 3D is proposed. The approach uses multi-scale features generated by deep learning. It comprises three steps: (1) extract single-scale deep features using 3D convolutional neural network (CNN); (2) subsample the input point cloud at multiple scales, with the point cloud at each scale being an input to the 3D CNN, and combine deep features at multiple scales to form multi-scale and hierarchical features; and (3) retrieve the probabilities that each point belongs to the intended semantic category using a softmax regression classifier. The proposed approach was tested against two publicly available point cloud datasets to demonstrate its performance and compared to the results produced by other existing approaches. The experiment results achieved 96.89% overall accuracy on the Oakland dataset and 91.89% overall accuracy on the Europe dataset, which are the highest among the considered methods.

Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network

ABSTRACTPoint cloud classification plays a critical role in many applications of airborne light detection and ranging (LiDAR) data. In this paper, we present a deep feature-based method for accurately classifying multiple ground objects from airborne LiDAR point clouds. With several selected attributes of LiDAR point clouds, our method first creates a group of multi-scale contextual images for each point in the data using interpolation. Taking the contextual images as inputs, a multi-scale convolutional neural network (MCNN) is then designed and trained to learn the deep features of LiDAR points across various scales. A softmax regression classifier (SRC) is finally employed to generate classification results of the data with a combination of the deep features learned from various scales. Compared with most of traditional classification methods, which often require users to manually define a group of complex discriminant rules or extract a set of classification features, the proposed method has the ability to automatically learn the deep features and generate more accurate classification results. The performance of our method is evaluated qualitatively and quantitatively using the International Society for Photogrammetry and Remote Sensing benchmark dataset, and the experimental results indicate that our method can effectively distinguish eight types of ground objects, including low vegetation, impervious surface, car, fence/hedge, roof, facade, shrub and tree, and achieves a higher accuracy than other existing methods.

APPLICATION OF SOFTMAX REGRESSION AND ITS VALIDATION FOR SPECTRAL-BASED LAND COVER MAPPING

Abstract. The presented Softmax Regression classifier is a generalization of logistic regression. It is used for multi-class classification, where classes are mutually exclusive. Implemented in a classification framework, it provides a flexible approach to customize a classification process. Traditional classification is focused with classifiers that can only be applied on the same dataset. The Softmax Regression classifier can be created and trained on a reference dataset using spectral and spatial information and then applied to similar data multiple times. We present the general workflow of Softmax Regression classification as part of a case study that is based on attribute images derived from hyperspectral airborne and elevation imagery.

Spectral-spatial classification of hyperspectral imagery using a dual-channel convolutional neural network

ABSTRACTIn this article, a novel dual-channel convolutional neural network (DC-CNN) framework is proposed for accurate spectral-spatial classification of hyperspectral image (HSI). In this framework, one-dimensional CNN is utilized to automatically extract the hierarchical spectral features and two-dimensional CNN is applied to extract the hierarchical space-related features, and then a softmax regression classifier is used to combine the spectral and spatial features together and predict classification results eventually. To overcome the problem of the limited available training samples in HSIs, we propose a simple data augmentation method which is efficient and effective for improving HSI classification accuracy. For comparison and validation, we test the proposed method along with three other deep-learning-based HSI classification methods on two real-world HSI data sets. Experimental results demonstrate that our DC-CNN-based method outperforms the state-of-the-art methods by a considerable margin.

Study of Sentiment Classification for Chinese Microblog Based on Recurrent Neural Network

The sentiment classification of Chinese Microblog is a meaningful topic. Many studies has been done based on the methods of rule and word-bag, and to understand the structure information of a sentence will be the next target. We proposed a sentiment classification method based on Recurrent neural network (RNN). We adopted the technology of distributed word representation to construct a vector for each word in a sentence; then train sentence vectors with fixed dimension for different length sentences with RNN, so that the sentence vectors contain both word semantic features and word sequence features; at last use softmax regression classifier in the output layer to predict each sentence's sentiment orientation. Experiment results revealed that our method can understand the structure information of negative sentence and double negative sentence and achieve better accuracy. The way of calculating sentence vector can help to learn the deep structure of sentence and will be valuable for different research area.

A novel method of diagnosing premature ventricular contraction based on sparse auto-encoder and softmax regression.

Premature ventricular contraction (PVC) is one of the most serious arrhythmias. Without early diagnosis and proper treatment, PVC can result in significant complications. In this paper, a novel feature extraction method based on a sparse auto-encoder (SAE) and softmax regression (SR) classifier was used to differentiate PVCs from other common Non-PVC rhythms, including normal sinus (N), left bundle branch block (LBBB), right bundle branch block (RBBB), atrial premature contraction (APC), and paced beat (PB) rhythms. The proposed method was analyzed using 40 ECG records obtained from the MIT-BIH Arrhythmia Database. The proposed method exhibited an overall accuracy of 99.4%, with a PVC recognition sensitivity and positive predictability of 97.9% and 91.8%, respectively.