Softmax Classifier Research Articles

Klasifikasi Jenis Jerawat Berdasarkan Citra Menggunakan Convolutional Neural Network dengan Arsitektur MobileNetV2

Among teenagers and adults worldwide, acne ranks as one of the most frequent skin concerns where each type of acne requires specific treatments. The proposed method in this study classifies five types of acne using deep learning. This methodology utilizes CNN architecture to specifically categorize different types of acne. The model used in this research adopts the MobileNetV2 architecture with transfer learning, consisting of several main layers: MobileNetV2 as the base model, Global Average Pooling, Flatten, Dense Layer, Dropout, and Softmax classification. The validation of the architecture utilizes secondary data consisting of 350 images. This dataset covers five different types of acne: Acne Fulminans, Acne Nodules, Papule, Pustule, and Fungal Acne, with each class containing 70 different samples. Challenges in automating this process include image lighting variations, image quality and data limitations. The CNN classification system shows performance rates of 89% for training and 80% for testing. The results indicatee that the proposed architecture can classify acne types with a reasonably good accuracy level, although there is room for improvement in model generalization.

AIGC Detection Model Based on Capsule Networks

With the advancement of technology, artificial intelligence-generated content (AIGC) has facilitated people's lives while also giving rise to numerous issues. Traditional AIGC detection methods have suffered from low accuracy and other problems, rendering them ineffective in detecting AI-generated images. Meanwhile, models trained on large datasets are constrained by the dataset size. Recent research has demonstrated that although training-free models are efficacious, their generalization ability poses a problem. In this paper, we propose a model based on capsule neural networks. The capsule network model acquires the spatial features of fake images and outputs image classification results via softmax classifier. We trained and evaluated the proposed AIGC image detection model using the publicly available MINIST dataset. The experimental results indicate that the capsule network-based model surpasses many traditional AIGC image detection models.

Fault diagnosis method of rolling bearing under variable operating conditions based on MFCCNN

A rolling bearing fault diagnosis method based on multi-scale feature fusion and cross-level connection convolutional neural network (MFCCNN) is proposed to address the issues of low accuracy and low generalization performance caused by the complex and variable actual working conditions of rolling bearings. Firstly, a Markov transition field (MTF) is used to convert a one-dimensional signal into a two-dimensional image with temporal correlation. Next, an improved channel attention mechanism is integrated into the multi-scale network to assign different weights to each channel information and extract initial features. Then, a position attention mechanism is created and incorporated into the cross-level connection to enhance the feature expression ability for deep feature extraction. The MFCCNN faults diagnosis model is developed based on the above module. Finally, the MTF images is input into the proposed model for training, and the Softmax classifier is utilized to achieve fault classification. To verify the effectiveness and superiority of the MFCCNN method, the CWRU and MFS rolling bearing dataset are selected for experimental verification. The results demonstrates that the MFCCNN model has higher diagnostic accuracy, stronger noise immunity, and generalization performance in varying operating conditions compared to other fault diagnosis methods.

Relational Extraction from Biomedical Texts with Capsule Network and Hybrid Knowledge Graph Embeddings

In the expanding landscape of biomedical literature, numerous latent associations outlined in scholarly papers await discovery and integration into biomedical databases. Biomedical Natural Language Processing (NLP) research focuses on automating knowledge extraction and mining from this literature, particularly emphasizing the essential task of Relation Extraction (RE). However, existing models have limitations, mainly in their applicability to partial datasets for RE tasks. Moreover, conventional models often treat RE as a binary classification challenge, which proves suboptimal given the diverse relationships, including intricate ones like similarity and hierarchy, present in the RE dataset. These limitations are exacerbated by the models’ inability to capture word-level positional nuances and sentence-level language features. In response to these challenges, we present a novel RE model called BicapBert. This model integrates neural networks and capsule networks, enhancing them with hybrid knowledge graph embeddings to extract relevant features. BicapBert utilizes PubMedBERT and capsule networks to extract word-level positional and sentence-level language features. It further captures knowledge features from a biomedical knowledge graph, integrating them with the aforementioned linguistic features. The amalgamated information is then input into a multi-layer perceptron, culminating in results derived through a softmax classifier. Experimental evaluations on three extensive RE task datasets showcase the state-of-the-art performance of our proposed model. Additionally, we validate the model’s efficacy on three randomly selected biomedical datasets for various tasks, further affirming its superiority.

Deep convolutional neural network model for classifying common bean leaf diseases

Common bean is one of the most important crops used by Ethiopian farmers for export and local consumption. However, the quality and quantity of this crop are heavily affected by different leaf diseases and affect crop growth. Currently, common bean disease detection is performed through expert visual observation. Disease detection through observation is costly, time-consuming, and inaccurate. As a result, in this paper, a novel deep convolutional neural network model is proposed for the automatic identification of common bean leaf diseases. This research mainly focuses on the identification of three common bean leaf diseases, such as common bean brown leaf spot, common bean leaf rust, and common bean leaf beetle. The proposed model was trained using a total of 1766 images of diseased and healthy common bean leaves. The proposed model has 12 convolutional layers. The classes of the diseases were classified using a softmax classifier. The proposed model achieved a training accuracy of 98%, a validation accuracy of 97.2%, and a testing accuracy of 96%. We retrained the pre-trained models (AlexNet, GoogleNet, VGG16) in a similar simulation environment to make a comparison with the proposed model. Therefore, the proposed model achieved better performance than the pre-trained models.

Fault diagnosis of HVCB via the subtraction average based optimizer algorithm optimized multi channel CNN-SABO-SVM network

The mechanical fault diagnosis of HVCB is important to ensure the stability of electric power systems. Aiming at the problem of poor diagnostic performance of deep learning methods under limited samples, this paper proposes an HVCB operating mechanism fault diagnosis model (multi-channel CNN-SABO-SVM, MCCSS) based on multimodal data fusion features and Subtraction-Average-Based Optimizer (SABO). This model extracts and fuses features from the input two-dimensional data using a multi-channel CNN network and then uses the multimodal data fusion features to diagnose HVCB faults. Additionally, the SVM is used instead of the Softmax classifier to classify the fused features of vibration and sound, compensating for the poor diagnostic performance and generalization ability of the CNN network in small sample data scenarios. To further enhance the fault diagnosis performance of the SVM, the SABO is introduced for hyperparameter optimization of the SVM classifier. An HVCB fault test platform was established to train and test the model with limited data. The experimental results show that, compared with the multi-channel CNN-SVM and the CNN model based on unimodal signals, the proposed multi-channel CNN-SABO-SVM model improves the accuracy by 2.66% and 10.66%, respectively, and effectively addresses the challenge of circuit breaker fault diagnosis with limited samples.

Magnetic resonance imaging diagnosis of ankle joint athletic injury based on machine learning algorithms

The diagnosis of ankle joint athletic injuries using traditional magnetic resonance imaging (MRI) relies on the subjective judgment and experience of doctors, and small structural changes in athletic injuries are difficult to accurately detect and diagnose. By using machine learning (ML) algorithms and image processing techniques to obtain objective and consistent diagnostic results, the accuracy of diagnosing ankle joint athletic injuries can be improved. This article collected a large number of MRI images of ankle joint athletic injuries, and preprocessed the collected images to extract morphological and texture features, and perform feature fusion. The Residual Network (ResNet) was improved, and the Leaky linear rectification function (ReLU, Corrected linear unit) activation function was introduced. The transfer learning was utilized to increase the convergence speed of the model, and the global maximum pooling layer and softmax classifier were used to construct the fully connected layer. After sufficient training on the training set, the findings on the test set indicated that the average accuracy of the improved ResNet model for ankle joint injury classification was 98.3%. The use of an improved ResNet model can effectively improve the diagnostic effectiveness of ankle joint athletic injuries, providing a new method for medical diagnosis of MRI.

A novel intelligent bearing fault diagnosis method based on image enhancement and improved convolutional neural network

Bearing fault diagnostic is crucial for ensuring the normal operation of equipment. However, the working environment of mechanical equipment is often very harsh, the collection of fault data is limited in the actual industrial process. The problem of insufficient fault samples occurs, which makes the feature extraction ability of the deep learning models drop dramatically. To solve this terrible problem, an intelligent diagnosis framework based on image enhancement and an improved convolutional neural network (IEICNN) is outlined in this article. First, a signal conversion method based on feature fusion is employed to convert multi-channel one-dimensional vibration signals into red − green − blue (RGB) images to obtain more comprehensive fault data representation. In addition, a new image enhancement method is proposed, which uses the improved Squeeze and Excitation-Residual Network (SE-ResNet) to learn and extract the advanced feature map, and uses deconvolution neural networks to reconstruct RGB image. Finally, the Softmax classifier is applied to identify the health conditions of the bearing. The superiority and robustness of IEICNN are validated using two bearing datasets. By comparing with several progressive comparison approaches, it is proven that the proposed approach has better robustness and effectiveness in the case of limited data scarcity.

A Mechanical Fault Identification Method for On-Load Tap Changers Based on Hybrid Time—Frequency Graphs of Vibration Signals and DSCNN-SVM with Small Sample Sizes

In engineering applications, the accuracy of on-load tap changer (OLTC) mechanical fault identification methods based on vibration signals is constrained by the quantity and quality of the samples. Therefore, a novel small-sample-size OLTC mechanical fault identification method incorporating short-time Fourier transform (STFT), synchrosqueezed wavelet transform (SWT), a dual-stream convolutional neural network (DSCNN), and support vector machine (SVM) is proposed. Firstly, the one-dimensional time-series vibration signals are transformed using STFT and SWT to obtain time–frequency graphs. STFT time–frequency graphs capture the global features of the OLTC vibration signals, while SWT time–frequency graphs capture the local features of the OLTC vibration signals. Secondly, these time–frequency graphs are input into the CNN to extract key features. In the fusion layer, the feature vectors from the STFT and SWT graphs are combined to form a fusion vector that encompasses both global and local time–frequency features. Finally, the softmax classifier of the traditional CNN is replaced with an SVM classifier, and the fusion vector is input into this classifier. Compared to the traditional fault identification methods, the proposed method demonstrates higher identification accuracy and stronger generalization ability under the conditions of small sample sizes and noise interference.

Fault diagnosis of floating offshore wind turbine based on bidirectional long short-term memory networks with sliding window

Owing to its ability to harvest more power in deeper waters than other types of offshore wind turbines, the floating offshore wind turbine (FOWT) has attracted significant interest from both academics and industry. However, deterioration in system performance and unscheduled shutdowns will significantly increase operation and maintenance costs. To avoid system damage and ensure operation safety, there is a great demand for developing FOWT fault diagnosis to detect and identify faults as early as feasible. Due to the enormous complexity of FOWT including dynamics and nonlinearity, effective FOWT fault diagnosis poses a significant challenge. This paper develops a novel FOWT fault diagnosis method using bidirectional long short-term memory (BiLSTM) with sliding window. Firstly, the dynamics are embedded by stacking time-dependent measurements using the sliding window technique. Then, the augmented data is fed into a BiLSTM network to exploit the nonlinearity of FOWT. Consequently, a softmax classifier is used for fault diagnosis. A high-fidelity 10 MW spar FOWT benchmark based on the NREL Fatigue, Aerodynamics, Structures, and Turbulence (FAST) model is used to validate the capability and effectiveness of the proposed method. Experimental results show that the proposed method outperforms other related methods in diagnosing critical faults in FOWTs under various operating conditions.

Convolutional neural networks method for folded naira currency denominations recognition and analysis

Sorting banknotes automatically in payment facilities or freewill donation boxes involves many tasks such as recognizing banknote denomination, fitness classification, and counterfeit detection. Different studies have addressed these problems using foreign currencies. In other words, very few researches have been conducted on naira banknote denomination recognition. A few studies did not consider folded naira banknote denomination recognition. To overcome this, we propose a Machine Learning Based Folded Banknote Denomination Recognition System. The proposed system automatically extracts relevant features and characteristics of the naira currency denomination in different positions, orientations, and folds. This automatic representation of the naira currency denomination enables the proposed system to reduce over-reliance on suggested characteristics, improve performance accuracy, and generalize the proposed system to new problems. Moreover, the proposed system was modeled using object-oriented analysis and design methodology, and implemented in the Keras framework interfaced with Tensorflow. Then, the Softmax classification algorithm was used to classify each Naira banknote denomination. The proposed machine learning-based folded banknote denomination recognition achieved an average accuracy of 95%, which shows the generalization ability of the proposed system to recognize folded banknote denominations in different orientations, positions, and folded forms.

Rolling bearing fault diagnosis method based on MSDCNN in strong noise environment

Aiming at the problems of poor noise resistance, high computational complexity and insufficient generalization performance of traditional bearing fault diagnosis methods based on deep learning, a rolling bearing fault diagnosis method based on multi-scale dynamic convolutional neural network (MSDCNN) is proposed. Firstly, the one-dimensional vibration signal of the rolling bearing is converted to the frequency domain by Fourier transform, and the features are further extracted by wide convolution kernel. Secondly, a multi-scale dynamic convolution structure is proposed, and the feature information extracted by convolution kernels of different sizes is given different weights by using an improved channel attention mechanism. Then, a self-calibrating spatial attention mechanism (SCSAM) is designed, and the extracted feature information is input into the spatial attention mechanism to capture the importance of different regions. Finally, the features are further extracted by small convolution kernels, and the fault category is classified by using Softmax classifier. The fault diagnosis performance of the proposed model is verified by three different data sets. The experimental results show that compared with other intelligent models, the proposed model has higher classification accuracy, better generalization ability and stronger robustness under strong noise background.

A Bullet Screen Sentiment Analysis Method That Integrates the Sentiment Lexicon with RoBERTa-CNN

Bullet screen, a form of online video commentary in emerging social media, is widely used on video websites frequented by young people. It has become a novel means of expressing emotions towards videos. The characteristics, such as varying text lengths and the presence of numerous new words, lead to ambiguous emotional information. To address these characteristics, this paper proposes a Robustly Optimized BERT Pretraining Approach (RoBERTa) + Convolutional Neural Network (CNN) sentiment classification algorithm integrated with a sentiment lexicon. RoBERTa encodes the input text to enhance semantic feature representation, and CNN extracts local features using multiple convolutional kernels of different sizes. Sentiment classification is then performed by a softmax classifier. Meanwhile, we use the sentiment lexicon to calculate the emotion score of the input text and normalize the emotion score. Finally, the classification results of the sentiment lexicon and RoBERTa+CNN are weighted and calculated. The bullet screens are grouped according to their length, and different weights are assigned to the sentiment lexicon based on their length to enhance the features of the model’s sentiment classification. The method combines the sentiment lexicon can be customized for the domain vocabulary and the pre-trained model can deal with the polysemy. Experimental results demonstrate that the proposed method achieves improvements in precision, recall, and F1 score. The experiments in this paper take the Russia–Ukraine war as the research topic, and the experimental methods can be extended to other events. The experiment demonstrates the effectiveness of the model in the sentiment analysis of bullet screen texts and has a positive effect on grasping the current public opinion status of hot events and guiding the direction of public opinion in a timely manner.

Research on Low-Voltage Arc Fault Based on CNN–Transformer Parallel Neural Network with Threshold-Moving Optimization

Low-voltage arc fault detection can effectively prevent fires, electric shocks, and other accidents, reducing potential risks to human life and property. The research on arc fault circuit interrupters (AFCIs) is of great significance for both safety in production scenarios and daily living disaster prevention. Considering the diverse characteristics of loads between the normal operational state and the arc fault condition, a parallel neural network structure is proposed for arc fault recognition, which is based on a convolutional neural network (CNN) and a Transformer. The network uses convolutional layers and Transformer encoders to process the low-frequency current and high-frequency components, respectively. Then, it uses Softmax classification to perform supervised learning on the concatenated features. The method combines the advantages of both networks and effectively reduces the required depth and computational complexity. The experimental results show that the accuracy of this method can reach 99.74%, and with the threshold-moving method, the erroneous judgment rate can be lower. These results indicate that the parallel neural network can definitely detect arc faults and also improve recognition efficiency due to its lean structure.

Early screening of miliary tuberculosis with tuberculous meningitis based on few-shot learning with multiple windows and feature granularities

Tuberculous meningitis (TBM) is a fatal tuberculosis caused by a large number of Mycobacterium tuberculosis (M. tuberculosis) spread by blood flow, with a case fatality rate of more than 50%. It is one of the most serious complications of miliary tuberculosis (MT), whose incidence is closely related to MT. If doctors can provide early diagnosis and active treatment for TBM, the case fatality rate will be significantly reduced. At present, there is a lack of methods to predict the progression of MT to TBM in clinic. To explore whether MT cases will experience TBM, we propose an early screening model of miliary tuberculosis with tuberculous meningitis (MT-TBM) based on few-shot learning with multiple windows and feature granularities (MWFG). This model aims to screen potential TBM cases through chest computerized tomography (CT) images of MT cases. Chest CT is a routine examination for MT cases. The MWFG module can extract more comprehensive features from a set of CT images of each MT case. The softmax classifier with adaptive regularization is trained on the cooperation of support set and query set, which can effectively prevent overfitting. Experiments on a dataset of 40 MT cases with chest CT images established by the medical records demonstrate that our proposed model achieves state-of-the-art performance in the early screening of MT-TBM. It can establish the connection between MT and MT-TBM through chest CT images of MT cases. The early screening model of MT-TBM based on few-shot learning with MWFG fills the research gap in computer-aided predicting TBM and has certain clinical effects. This research can provide some reference for clinicians in early diagnosis of MT-TBM and help clinicians in the early prevention and treatment of TBM for MT patients.

Rail State Recognition Method Based on a Convolutional Block Attention Module

Abstract In order to solve the problems of high subjectivity and serious lag in traditional rail surface state recognition based on manual experience, a rail surface state recognition model based on convolution attention is proposed. First, the transfer learning method is used to pre-train the ImageNet data set to obtain the model parameters. Secondly, the 3×3 convolution in the ResNet-50 residual block is replaced with a convolutional block attention module (CBAM) to obtain a new CBAM-ResNet. Finally, the model identification results of the rail surface status are obtained through the Softmax classifier. The results show that this method can effectively identify the rail surface status, and the identification accuracy can reach 99.68%, which is better than other deep neural network models.

A multi-sensor fused incremental detection model for blade crack with cross-attention mechanism and Dempster-Shafer evidence theory

Deep learning-based blade crack detection models work on the premise of a fixed data distribution, while the influx of new dataset for faults under blade crack propagation often leads to a catastrophic forgetting problem. Meanwhile, it is difficult for a single sensor to reflect the health status of the blade comprehensively under the limitation of installation location and coverage. To solve the above problems, a multi-sensor fused incremental detection model (MFIDM) for blade cracks with the cross-attention mechanism and the Dempster-Shafer evidence theory (DST) is proposed. Firstly, vibration signals of centrifugal fans are collected by multiple accelerometers deployed at different locations. Then, a two-branch feature fusion method based on the cross-attention mechanism is proposed to overcome the class imbalance due to the replay incremental learning method. After that, the fused features are fed into a Softmax classifier to complete the initial classification of blade status. Finally, a modified DST based on the cross-correlation energy is adopted for multi-sensor decision fusion to obtain the final blade crack detection results. The effectiveness of the proposed method is verified by two incremental blade crack datasets, and MFIDM achieves the better performance compared with other related incremental detection methods.

A hybrid features fusion-based framework for classification of breast micronodules using ultrasonography

BackgroundBreast cancer is one of the leading diseases worldwide. According to estimates by the National Breast Cancer Foundation, over 42,000 women are expected to die from this disease in 2024.ObjectiveThe prognosis of breast cancer depends on the early detection of breast micronodules and the ability to distinguish benign from malignant lesions. Ultrasonography is a crucial radiological imaging technique for diagnosing the illness because it allows for biopsy and lesion characterization. The user’s level of experience and knowledge is vital since ultrasonographic diagnosis relies on the practitioner’s expertise. Furthermore, computer-aided technologies significantly contribute by potentially reducing the workload of radiologists and enhancing their expertise, especially when combined with a large patient volume in a hospital setting.MethodThis work describes the development of a hybrid CNN system for diagnosing benign and malignant breast cancer lesions. The models InceptionV3 and MobileNetV2 serve as the foundation for the hybrid framework. Features from these models are extracted and concatenated individually, resulting in a larger feature set. Finally, various classifiers are applied for the classification task.ResultsThe model achieved the best results using the softmax classifier, with an accuracy of over 95%.ConclusionComputer-aided diagnosis greatly assists radiologists and reduces their workload. Therefore, this research can serve as a foundation for other researchers to build clinical solutions.

Impact Load Localization Based on Multi-Scale Feature Fusion Convolutional Neural Network.

In order to achieve impact load localization of complex structures such as ships, this paper proposes a multi-scale feature fusion convolutional neural network (MSFF-CNN) method for impact load localization. An end-to-end machine learning model is used, where the raw vibration signals of impact loads are directly fed into the network model to avoid the process of feature extraction. Automatic feature learning and feature concatenation of the signal are achieved through four independent convolutional layers, each using a different size of convolutional kernel. Data normalization and L2 regularization techniques are introduced to enhance the data and prevent overfitting. Classification and localization of impact loads are accomplished using a softmax classification layer. Validation experiments are carried out using a ship's stern compartment model. Our results show that the classification and localization accuracy of the impact load sample group of MSFF-CNN reaches 94.29% compared with a traditional CNN. The method further improves the ability of the network to extract state features, takes local perception and global vision into account, effectively improves the classification ability of the model, and has good prospects for engineering applications.

SAGB: self-attention with gate and BiGRU network for intrusion detection

Network traffic intrusion detection technology plays an important role in host and platform security. At present, machine learning and deep learning methods are often used for network traffic intrusion detection. However, the imbalance of relevant data sets will cause the model algorithm to learn the features of the majority categories and ignore the features of the minority categories, which will seriously affect the precision of network intrusion detection models. The number of samples of the attacks is much less than the number of normal samples. The classification performance on unbalanced data sets is poor and can not identify the minority attack samples well. To solve these problems, this paper proposed the resampling mechanism, which used random undersampling for the majority categories samples and K-Smote oversampling for the minority categories samples, so as to generate a more balanced data set and improve the model's detection rate for the minority categories. This paper proposed the Self-Attention with Gate (SAG) and BiGRU network model for intrusion detection on the CICIDS2017 data set, which can fully extract high-dimensional information from data samples and improve the detection rate of network attacks. The Self-Attention with Gate mechanism (SAG) based on the Transformer performed the feature extraction, filtered the irrelevant noise information, then extracted the long-distance dependency temporal sequence features by the BiGRU network, and obtained the classification results through the SoftMax classifier. Compared to the experimental results of other algorithms, such as Random Forest (RF) and BiGRU, it can be found that the overall classification precision for the SAG-BiGRU model in this paper is much higher and also has a good effect on the NSL-KDD data set.