Support Vector Machine Network Research Articles

Research on fault diagnosis of electric submersible pump based on improved convolutional neural network with Bayesian optimization

A conventional intelligent fault diagnosis approach for electric submersible pumps (ESPs) is heavily dependent on manual expertise for feature extraction. Meanwhile, a conventional convolutional neural network (CNN) exhibits an excess of parameters and requires a substantial volume of training samples. In this paper, a fault diagnosis algorithm of ESPs based on a Bayesian optimization-one-dimensional convolutional neural network-support vector machine (BO-1DCNN-SVM) is proposed by combining a 1DCNN with a SVM and using the algorithm BO to tune the improved 1DCNN. First, the method uses the self-extracting feature capability of the 1DCNN to solve the problem that traditional diagnosis methods over-rely on manual experience extraction. Meanwhile, the last layer of the convolutional neural network Softmax layer is replaced by the SVM to effectively process a few sample data. The accuracy and generalization ability of the fault classification of the electric submersible pump are improved. Then, the Bayesian optimization algorithm is used to find the optimal combination of hyperparameters for the improved 1DCNN-SVM model to further improve the prediction performance of the fault diagnosis model. Finally, the experimental results achieved a classification accuracy of 96.64%, which is 5% higher than existing CNN approaches with data samples of similar scale in this paper. The proposed method also proved to be highly accurate and robust in fault diagnosis.

Identification of two-phase flow patterns based on capacitance data of electrical capacitance tomography with semi-supervised generative adversarial network.

Currently, the flow pattern identification algorithms based on ECT (electrical capacitance tomography) technology have low identification accuracy for complex flow patterns and require a large amount of label data for learning. A novel flow pattern identification method based on a semi-supervised generative adversarial network (SGAN) with capacitance data of ECT is proposed. First, the principles of the ECT technique and general GAN are briefly described, and the model parameters, loss function, and training process of the SGAN are explained in detail. Second, a capacitance data sample set of 11 400 random flow patterns is constructed by co-simulations of COMSOL and MATLAB, and then, the SGAN and BP (back propagation) and SVM (support vector machine) network models are trained and validated by the training set. Finally, static experiments are conducted on the self-developed ECT system, and the identification results of different algorithms are compared and analyzed by modifying the label sample size of the training set. The experimental results show that SGAN maintains a higher average identification accuracy under the training condition where the number of label samples of SGAN is ten times smaller than that of the other two algorithms.

Machine learning prediction models for different stages of non-small cell lung cancer based on tongue and tumor marker: a pilot study

ObjectiveTo analyze the tongue feature of NSCLC at different stages, as well as the correlation between tongue feature and tumor marker, and investigate the feasibility of establishing prediction models for NSCLC at different stages based on tongue feature and tumor marker.MethodsTongue images were collected from non-advanced NSCLC patients (n = 109) and advanced NSCLC patients (n = 110), analyzed the tongue images to obtain tongue feature, and analyzed the correlation between tongue feature and tumor marker in different stages of NSCLC. On this basis, six classifiers, decision tree, logistic regression, SVM, random forest, naive bayes, and neural network, were used to establish prediction models for different stages of NSCLC based on tongue feature and tumor marker.ResultsThere were statistically significant differences in tongue feature between the non-advanced and advanced NSCLC groups. In the advanced NSCLC group, the number of indexes with statistically significant correlations between tongue feature and tumor marker was significantly higher than in the non-advanced NSCLC group, and the correlations were stronger. Support Vector Machine (SVM), decision tree, and logistic regression among the machine learning methods performed poorly in models with different stages of NSCLC. Neural network, random forest and naive bayes had better classification efficiency for the data set of tongue feature and tumor marker and baseline. The models’ classification accuracies were 0.767 ± 0.081, 0.718 ± 0.062, and 0.688 ± 0.070, respectively, and the AUCs were 0.793 ± 0.086, 0.779 ± 0.075, and 0.771 ± 0.072, respectively.ConclusionsThere were statistically significant differences in tongue feature between different stages of NSCLC, with advanced NSCLC tongue feature being more closely correlated with tumor marker. Due to the limited information, single data sources including baseline, tongue feature, and tumor marker cannot be used to identify the different stages of NSCLC in this pilot study. In addition to the logistic regression method, other machine learning methods, based on tumor marker and baseline data sets, can effectively improve the differential diagnosis efficiency of different stages of NSCLC by adding tongue image data, which requires further verification based on large sample studies in the future.

Interpreting coal component content in logging data by combining gray relational analysis and hybrid neural network

The coal component content is an important parameter during the coal resources exploration and exploitation. Previous logging curve regression and single neural network methods have the disadvantages of low accuracy and weak generalization ability in calculating coal component content. In this study, a gray relational analysis-hybrid neural network (GRA-HNN) method is developed by combining GRA and HNN to predict coal component content in logging data. First, the correlation degree between different conventional logging data and coal components is calculated using the GRA method, and logging curves with a correlation degree of ≥0.7 are selected as the input training data set. Then, a back propagation neural network, support vector machine neural network, and radial basis function neural network of different coal components are constructed based on the selected optimal input logging data, and the weighted average strategy is used to form an HNN prediction model. Finally, the GRA-HNN method is used to predict the coal component content of coalbed methane production wells in the Panji mining area. The application results indicate that the coal component content predicted by the GRA-HNN method has the highest accuracy compared with the logging curve regression method and its single neural network model, with a maximum average relative error of 13.4%. In addition, the accuracy of coal component content predicted by some single intelligent models is not always higher than the logging curve regression method, indicating that the neural network model is not necessarily suitable for all coal component content predictions. Our GRA-HNN method not only optimizes the prediction performance of a single neural network model by selecting effective input parameters but also comprehensively considers the prediction effect of several neural network models, which strengthens the generalization ability of neural network model and increases the log interpretation accuracy of coal component content.

Analysis of WSI Images by Hybrid Systems with Fusion Features for Early Diagnosis of Cervical Cancer.

Cervical cancer is one of the most common types of malignant tumors in women. In addition, it causes death in the latter stages. Squamous cell carcinoma is the most common and aggressive form of cervical cancer and must be diagnosed early before it progresses to a dangerous stage. Liquid-based cytology (LBC) swabs are best and most commonly used for cervical cancer screening and are converted from glass slides to whole-slide images (WSIs) for computer-assisted analysis. Manual diagnosis by microscopes is limited and prone to manual errors, and tracking all cells is difficult. Therefore, the development of computational techniques is important as diagnosing many samples can be done automatically, quickly, and efficiently, which is beneficial for medical laboratories and medical professionals. This study aims to develop automated WSI image analysis models for early diagnosis of a cervical squamous cell dataset. Several systems have been designed to analyze WSI images and accurately distinguish cervical cancer progression. For all proposed systems, the WSI images were optimized to show the contrast of edges of the low-contrast cells. Then, the cells to be analyzed were segmented and isolated from the rest of the image using the Active Contour Algorithm (ACA). WSI images were diagnosed by a hybrid method between deep learning (ResNet50, VGG19 and GoogLeNet), Random Forest (RF), and Support Vector Machine (SVM) algorithms based on the ACA algorithm. Another hybrid method for diagnosing WSI images by RF and SVM algorithms is based on fused features of deep-learning (DL) models (ResNet50-VGG19, VGG19-GoogLeNet, and ResNet50-GoogLeNet). It is concluded from the systems' performance that the DL models' combined features help significantly improve the performance of the RF and SVM networks. The novelty of this research is the hybrid method that combines the features extracted from deep-learning models (ResNet50-VGG19, VGG19-GoogLeNet, and ResNet50-GoogLeNet) with RF and SVM algorithms for diagnosing WSI images. The results demonstrate that the combined features from deep-learning models significantly improve the performance of RF and SVM. The RF network with fused features of ResNet50-VGG19 achieved an AUC of 98.75%, a sensitivity of 97.4%, an accuracy of 99%, a precision of 99.6%, and a specificity of 99.2%.

Fault Diagnosis of Electric Submersible Pumps Using a Three‐Stage Multiscale Feature Transformation Combined with CNN–SVM

A convolutional neural network support vector machine (CNN–SVM) method based on multichannel feature fusion is used for progressive fault diagnosis of offshore oil and gas wells. The excellent classification performance of CNN is attributed to its ability to extract feature representations from large amounts of easily distinguishable data. However, the capability of CNN is severely constrained by the noisy and small sample amount of the electric submersible pump fault data to be studied in this article. First, 12 representative statistical features are extracted from the raw data to reduce the noise. Then, the feature mapping model is designed based on CNN migration learning. Finally, SVM is used instead of softmax function to adopt representative features directly from the mapping model for fault classification. Comparative experimental results show that the accuracy of fault diagnosis using feature‐extracted samples is better than using the raw samples directly. The proposed CNN–SVM approach has the best classification results compared to SVM, BPNN, CNN, BPNN–SVM, CNN–Attention, CNN–LSTM, and CNN–LSTM–Attention, which implies that manual feature extraction is still an indispensable tool in the fault diagnosis process.

Wavelet Based Detection and Classification Power Quality Disturbance using SVM and PSO

This paper introduces a novel approach to detect and classify power quality disturbance in the power system using Support Vector Machine (SVM). The proposed method requires less number of features as compared to conventional approach for the identification. For the classification, 8 types of disturbances are taken in to account. The classification performance of SVM is compared with Radial basis Function neural network (RBNN).The classification accuracy of the SVM network is improved, just by rewriting the weights and updating the weights with the help of cognitive as well as the social behaviour of particles along with fitness value by using Particle Swarm Optimization (PSO). The simulation results possess significant improvement over existing methods in signal detection and classification with lesser number of features

Inspection and classification of wheat quality using image processing

Wheat plays an important role in our daily life and industrial production. Several computer vision approaches have been proposed for classifying wheat quality, but there were some methods focusing on the problem of cohesive wheats while image processing. In this paper, we designed a single kernel guide groove to separate the cohesive wheats, which could simplify the algorithm of image processing and improve the accuracy rate of classification. For the method followed while recording the data, the image information must be converted into digital information, and the results are provided using appropriate image processing algorithms. Image preprocessing steps such as binarization, image enhancement, image segmentation, and morphological processing were used to reduce noise. For image segmentation, we proposed the following new segmentation methods: (1) extracting wheat region by converting image to H channel and (2) watershed algorithm based on Euclidean distance transformation. For the classification model, 22 features of 7 different qualities of wheat were inputted in the Back Propagation (BP) neural network and Support Vector Machine (SVM) model, and the overall correct classification rates were determined to be 91% and 97% for SVM and BP neural network, respectively. The BP neural network was more suitable for wheat appearance quality detection.

Emotional speech Recognition using CNN and Deep learning techniques

Emotions are considered an integral and vital part of human life. It acts as a means of expressing one's opinions and informing others about one's bodily and emotional well-being. The Speech Emotion Recognition (SER) system is used to extract and predict the emotional tone of a speaker through audio signals. Universally, emotions can be often categorized and grouped into mere types like Anger, Happiness, Sadness, and even Neutral emotional state. One can then use this as a criterion to develop a system provided, they are equipped with finite and required resources. Then through proper training and procedure, the system can be made to recognize the speaker’s emotional state. In these situations, speech emotion detection is performed using spectral and prosodic capabilities because each of those aspects includes the essential data to determine the speaker's emotional state. Mel-frequency Cepstral Coefficients (MFCC) are one of the spectral features that can be used to detect the speaker’s emotional state. Frequency, loudness, and pitch of a sound signal are some of the many available prosodic features that would then be able to help train and construct a machine-learning model that can uniquely identify and differentiate between the underlying emotions present in a given speech signal. It is known that the pitch of an audio signal is unique and can be used as a measure to easily distinguish between different audio signals. They can be detected from the selected features, allowing us to classify the gender type of the speaker. Support Vector Machines (SVM) are supervised learning models that examine the data for regression and classification in machine learning. They generally identify and classify the gender of the speaker in the case of a Speech Emotion Recognition system. Furthermore, some studies rely on Radial-Basis Function (RBF) and Back Propagation networks. They are capable enough to be used to identify and recognize the human emotions in the signal constrained on specific and selected features. This study presents a Speech Emotion Recognition system that outperforms an existing system in terms of data, feature selection, and methodology. Its goal is to identify speech precepts based on emotions more accurately, on average 78% accurately, and with fewer false positives. The paper aims to discuss the importance of emotions in human communication and introduce the concept of Speech Emotion Recognition, which extracts and predicts the emotional tone of a speaker through audio signals. It explains how spectral and prosodic features can be used to identify the emotional state of the speaker, with MFCC being one of the commonly used spectral features. Prosodic features such as pitch, loudness, and frequency can also be used to identify gender and differentiate between emotions in speech signals. SVM, RBF, and Back Propagation networks are machine learning models used in speech emotion recognition; these more advanced models outperform existing systems in data collection and feature selection methodology, achieving an accuracy rate of 78% with fewer false positives.

Identifying Young Adults at High Risk for Weight Gain Using Machine Learning

IntroductionWeight gain among young adults continues to increase. Identifying adults at high risk for weight gain and intervening before they gain weight could have a major public health impact. Our objective was to develop and test electronic health record-based machine learning models to predict weight gain in young adults with overweight/class 1 obesity. MethodsSeven machine learning models were assessed, including three regression models, random forest, single-layer neural network, gradient-boosted decision trees, and support vector machine (SVM) models. Four categories of predictors were included: 1) demographics; 2) obesity-related health conditions; 3) laboratory data and vital signs; and 4) neighborhood-level variables. The cohort was split 60:40 for model training and validation. Area under the receiver operating characteristic curves (AUC) were calculated to determine model accuracy at predicting high-risk individuals, defined by ≥ 10% total body weight gain within 2 y. Variable importance was measured via generalized analysis of variance procedures. ResultsOf the 24,183 patients (mean [SD] age, 32.0 [6.3] y; 55.1% females) in the study, 14.2% gained ≥10% total body weight. Area under the receiver operating characteristic curves varied from 0.557 (SVM) to 0.675 (gradient-boosted decision trees). Age, sex, and baseline body mass index were the most important predictors among the models except SVM and neural network. ConclusionsOur machine learning models performed similarly and had modest accuracy for identifying young adults at risk of weight gain. Future models may need to incorporate behavioral and/or genetic information to enhance model accuracy.

Research and application of artificial intelligence in disease prediction

Now, more and more diseases can be predicted by AI algorithms or early disease prediction, enabling patients to prepare for two kinds of physics. Because the utilitarian model of disease prediction is crucial for mankind to advance in medicine. We consider SVM(Support Vector Machine), random forest and neural networkCNNto be the proper methods to do the prediction based on the symptoms of Four diseases. SVM and Neural Network show better accuracy in the training. However, Random Forest has a slightly lower accuracy. We hope our method can be helpful in AI field.

An Atrial Fibrillation Classification Method Study Based on BP Neural Network and SVM

Atrial fibrillation is a common arrhythmia, and its diagnosis is interfered by many factors. In order to achieve applicability in diagnosis and improve the level of automatic analysis of atrial fibrillation to the level of experts, the automatic detection of atrial fibrillation is very important. This study proposes an automatic detection algorithm for atrial fibrillation based on BP neural network (back propagation network) and support vector machine (SVM). The electrocardiogram (ECG) segments in the MIT-BIH atrial fibrillation database are divided into 10, 32, 64, and 128 heartbeats, respectively, and the Lorentz value, Shannon entropy, K-S test value and exponential moving average value are calculated. These four characteristic parameters are used as the input of SVM and BP neural network for classification and testing, and the label given by experts in the MIT-BIH atrial fibrillation database is used as the reference output. Among them, the use of atrial fibrillation in the MIT-BIH database, the first 18 cases of data are used as the training set, and the last 7 cases of data are used as the test set. The results show that the accuracy rate of 92% is obtained in the classification of 10 heartbeats, and the accuracy rate of 98% is obtained in the latter three categories. The sensitivity and specificity are both above 97.7%, which has certain applicability. Further validation and improvement in clinical ECG data will be done in next study.

Hyperspectral Estimation of Nitrogen Content in Wheat Based on Fractional Difference and Continuous Wavelet Transform

Nitrogen content is a crucial index for crop growth diagnosis and the exact estimation of nitrogen content is of great significance for grasping crop growth status in real-time. This paper takes winter wheat as the study object and the precision agriculture demonstration area of the Jiaozuo Academy of Agricultural and Forestry Sciences in Henan Province as the research area. The hyperspectral reflectance data of the wheat canopy in different growth periods are obtained with the ASD ground object hyperspectral instrument, and the original canopy spectral data are preprocessed by fractional differential and continuous wavelet transform; then, the vegetation indices are established, correlation analysis with nitrogen content is conducted, and the fractional differential spectra are selected; finally, based on the wavelet energy coefficient and the vegetation indices with strong correlations, the methods of support vector machine (SVM), ridge regression, stepwise regression, Gaussian process regression (GPR), and the BP neural network are used to construct the estimation model for nitrogen content in wheat at different growth stages. By adopting the R2 and root mean square error (RMSE) indices, the best nitrogen content estimation model at every growth stage is selected. The overall analysis of the nitrogen content estimation effect indicated that for the four growth periods, the maximum modeling and validation R2 of the nitrogen content estimation models of the SVM, ridge regression, stepwise regression, GPR, and BP neural network models reached 0.95 and 0.93, the average reached 0.76 and 0.71, and the overall estimation effect was good. The average values of the modeling and validation R2 of the nitrogen content estimation model at the flag picking stage were 0.85 and 0.81, respectively, which were 37.10% and 44.64%, 1.19% and 3.85%, and 14.86% and 17.39% higher than those at the jointing stage, flowering stage, and filling stage, respectively. Therefore, the model of the flag picking stage has higher estimation accuracy and a better estimation effect on the nitrogen content. For the different growth stages, the optimal estimation models of nitrogen content were different. Among them, continuous wavelet transform combined with the BP neural network model can be the most effective method for estimating the N content in wheat at the flagging stage. The paper provides an effective method for estimating the nitrogen content in wheat and a new idea for crop growth monitoring.

Automatic Detection of Daytime Sea Fog Based on Supervised Classification Techniques for FY-3D Satellite

Polar-orbiting satellites have been widely used for detecting sea fog because of their wide coverage and high spatial and spectral resolution. FengYun-3D (FY-3D) is a Chinese satellite that provides global sea fog observation. From January 2021 to October 2022, the backscatter and virtual file manager products from CALIPSO (Cloud-Aerosol Lidar and Infrared Pathfinder Satellite Observation) were used to label samples of different atmospheric conditions in FY-3D images, including clear sky, sea fog, low stratus, fog below low stratus, mid–high-level clouds, and fog below the mid–high-level clouds. A 13-dimensional feature matrix was constructed after extracting and analyzing the spectral and texture features of these samples. In order to detect daytime sea fog using a 13-dimensional feature matrix and CALIPSO sample labels, four supervised classification models were developed, including Decision Tree (DT), Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Neural Network. The accuracy of each model was evaluated and compared using a 10-fold cross-validation procedure. The study found that the SVM, KNN, and Neural Network performed equally well in identifying low stratus, with 85% to 86% probability of detection (POD). As well as identifying the basic components of sea fog, the SVM model demonstrated the highest POD (93.8%), while the KNN had the lowest POD (92.4%). The study concludes that the SVM, KNN, and Neural Network can effectively distinguish sea fog from low stratus. The models, however, were less effective at detecting sub-cloud fog, with only 11.6% POD for fog below low stratus, and 57.4% POD for fog below mid–high-level clouds. In light of this, future research should focus on improving sub-cloud fog detection by considering cloud layers.

Cooperative spectrum sensing based hybrid machine learning technique for prediction of secondary user in cognitive radio networks

The spectrum scarcity problem in today’s wireless communication network is addressed through the use of a cognitive radio network (CRN). Detection in the spectrum is made easier by cooperative spectrum sensing (CSS), which is a tool developed by the military. The fusion centre receives the sensing information from each secondary user and uses it to make a global conclusion about the presence of the principal user. Literature has offered several different methods for decision making that lack scalability and robustness. CSS censoring is inspected in the attendance of faded settings in the current study. Rayleigh fading, which affects reporting channels (R), is examined in detail. Multiple antennae and an energy detector (ED) are used by each secondary user (SU). A selection combiner (SC) combines the ED outputs with signals from the primary user (PU), which are established by several antennas on SU, before the joint signal is utilised to make a local result. SUs are expurgated at the fusion centre (FC) using a hybrid Support Vector Machine (SVM) that significantly improves detection performance and reduces the number of false positives. With a minimum false alarm probability of 0.1, error rate of 0.04, spectrum utilization of 99%, throughput of 2.9kbps and accuracy of 99%, proposed model attains better performance than standard SVM and Artificial Neural Network (ANN) models.

A Bio-Inspired Spiking Neural Network with Few-Shot Class-Incremental Learning for Gas Recognition.

The sensitivity and selectivity profiles of gas sensors are always changed by sensor drifting, sensor aging, and the surroundings (e.g., temperature and humidity changes), which lead to a serious decline in gas recognition accuracy or even invalidation. To address this issue, the practical solution is to retrain the network to maintain performance, leveraging its rapid, incremental online learning capacity. In this paper, we develop a bio-inspired spiking neural network (SNN) to recognize nine types of flammable and toxic gases, which supports few-shot class-incremental learning, and can be retrained quickly with a new gas at a low accuracy cost. Compared with gas recognition approaches such as support vector machine (SVM), k-nearest neighbor (KNN), principal component analysis (PCA) +SVM, PCA+KNN, and artificial neural network (ANN), our network achieves the highest accuracy of 98.75% in five-fold cross-validation for identifying nine types of gases, each with five different concentrations. In particular, the proposed network has a 5.09% higher accuracy than that of other gas recognition algorithms, which validates its robustness and effectiveness for real-life fire scenarios.

To Establish an Early Prediction Model for Acute Respiratory Distress Syndrome in Severe Acute Pancreatitis Using Machine Learning Algorithm.

To develop binary and quaternary classification prediction models in patients with severe acute pancreatitis (SAP) using machine learning methods, so that doctors can evaluate the risk of patients with acute respiratory distress syndrome (ARDS) and severe ARDS at an early stage. A retrospective study was conducted on SAP patients hospitalized in our hospital from August 2017 to August 2022. Logical Regression (LR), Random Forest (RF), Support Vector Machine (SVM), Decision Tree (DT), and eXtreme Gradient Boosting (XGB) were used to build the binary classification prediction model of ARDS. Shapley Additive explanations (SHAP) values were used to interpret the machine learning model, and the model was optimized according to the interpretability results of SHAP values. Combined with the optimized characteristic variables, four-class classification models, including RF, SVM, DT, XGB, and Artificial Neural Network (ANN), were constructed to predict mild, moderate, and severe ARDS, and the prediction effects of each model were compared. The XGB model showed the best effect (AUC = 0.84) in the prediction of binary classification (ARDS or non-ARDS). According to SHAP values, the prediction model of ARDS severity was constructed with four characteristic variables (PaO2/FiO2, APACHE II, SOFA, AMY). Among them, the overall prediction accuracy of ANN is 86%, which is the best. Machine learning has a good effect in predicting the occurrence and severity of ARDS in SAP patients. It can also provide a valuable tool for doctors to make clinical decisions.

A joint method of spatial–spectral features and BP neural network for hyperspectral image classification

A hyperspectral image (HSI) has also highly correlated and redundant data, in addition to abundant spatial and spectral information. Only the spectral characteristics were usually utilized to perform the HSI classification in most previous studies, which leads to unsatisfactory accuracy and precision. The combination of Principal Component Analysis (PCA), Local Binary Pattern (LBP) and Back Propagation Neural Network (BPNN) (PCA-LBP-BPNN) was used to propose a novel classification method. More specifically, PCA was first used to reduce the dimensionality of HSIs for obtaining independent spectral bands sensitive to the classified objects. LBP was then adopted to extract the spatial texture features. Finally, the feature vectors were formed by fusing spatial–spectral features and input into BPNN for HSI classification. Three key parameters including the number of principal components (p) and the neurons in the hidden layer (l) as well as the learning rate (r) were optimally selected to improve the classification accuracy. Three publicly available hyperspectral datasets including Pavia University (PU), Salinas (Sa) and Botswana (Bo) were selected to validate the performance by comparing the kNN (k-Nearest Neighbor), SVM (Support Vector Machine) and Contextual Deep Convolutional Neural Network (CDCNN). The overall accuracy of PCA-LBP-BPNN reached 93.67%, 98.09% and 92.97%, respectively, for the three datasets. The method had a satisfying performance than kNN, SVM and CDCNN for the PU and Sa, but it had lower accuracies than kNN and CDCNN for the Bo due to extremely similar spectral responses. PCA-LBP-BPNN generally has satisfactory practicability and robustness in adapting to different hyperspectral datasets.

Leaf-based Classification of Important Indigenous Tree Species by Different Feature Extraction Techniques and Selected Classification Algorithms

The machine learning algorithms, namely, k-Nearest Neighbor (KNN), Support Vector Machine (SVM), Back-Propagation (BP) networks, and Convolutional Neural Networks (CNN) are four of the mostly used classifiers. Different sets of features are required as input in different application domains. In this paper, a set of significant leaf features and classification model was determined with a high accuracy in classifying important indigenous tree species. Leaf images were acquired using a scanner to control the image quality. The image dataset was then duplicated into two sets. The first set was labeled with their correct classes, preprocessed, and segmented in preparation for feature extraction. The leaf features extracted were leaf shape, leaf color, and leaf texture. Then, training and classification was done by KNN, SVM, and BP networks. On the other hand, the second set was unlabeled for training and classification by CNN. A CNN model was built and chosen with the best training and validation accuracy and the least training and validation loss rate. The study concluded that using all three leaf features for classification by BP networks resulted in a 93.48% accuracy with training done by supervised learning. However, the CNN achieved a high accuracy rate of 98.5% making it the best approach for classification of tree species using digital leaf images in the context of this study.

Analyzing and Explaining Black-Box Models for Online Malware Detection

In recent years, a significant amount of research has focused on analyzing the effectiveness of machine learning (ML) models for malware detection. These approaches have ranged from methods such as decision trees and clustering to more complex approaches like support vector machine (SVM) and deep neural networks. In particular, neural networks have proven to be very effective in detecting complex and advanced malware. This, however, comes with a caveat. Neural networks are notoriously complex. Therefore, the decisions that they make are often just accepted without questioning why the model made that specific decision. The black box characteristic of neural networks has challenged researchers to explore methods to explain black-box models such as SVM and neural networks and their decision-making process. Transparency and explainability give the experts and malware analysts assurance and trustworthiness about the ML models’ decisions. In addition, it helps in generating comprehensive reports that can be used to enhance cyber threat intelligence sharing. As such, this much needed analysis drives our work in this paper to explore the explainability and interpretability of ML models in the field of online malware detection. In this paper, we used the Shapley Additive exPlanations (SHAP) explainability technique to achieve efficient performance in interpreting the outcome of different ML models such as SVM Linear, SVM-RBF (Radial Basis Function), Random Forest (RF), Feed-Forward Neural Net (FFNN), and Convolutional Neural Network (CNN) models trained on an online malware dataset. To explain the output of these models, explainability techniques such as KernalSHAP, TreeSHAP, and DeepSHAP are applied to the obtained results.