Performance Of Classification Models Research Articles

Exploring the impact of noise and degradations on heart sound classification models

The development of data-driven heart sound classification models has been an active area of research in recent years. To develop such data-driven models in the first place, heart sound signals need to be captured using a signal acquisition device. However, it is almost impossible to capture noise-free heart sound signals due to the presence of internal and external noises in most situations. Such noises and degradations in heart sound signals can potentially reduce the accuracy of data-driven classification models. Although different techniques have been proposed in the literature to address the noise issue, how and to what extent different noise and degradations in heart sound signals impact the accuracy of data-driven classification models remains unexplored. To answer this question, we produced a synthetic heart sound dataset including normal and abnormal heart sounds contaminated with a variety of noise and degradations. We used this dataset to investigate the impact of noise and degradation in heart sound recordings on the performance of different classification models. The results show different noises and degradations affect the performance of heart sound classification models to a different extent; some are more problematic for classification models, and others are less destructive. Comparing the findings of this study with the results of a survey we previously carried out with a group of clinicians shows noise and degradations that are more detrimental to classification models are also more disruptive to accurate auscultation. The findings of this study can be leveraged to develop targeted heart sound quality enhancement approaches — which adapt the type and aggressiveness of quality enhancement based on the characteristics of noise and degradation in heart sound signals.

Fuzzy Discretization on the Multinomial Naïve Bayes Method for Modeling Multiclass Classification of Corn Plant Diseases and Pests

As an agricultural commodity, corn functions as food, animal feed, and industrial raw material. Therefore, diseases and pests pose a major challenge to the production of corn plants. Modeling the classification of corn plant diseases and pests based on digital images is essential for developing an information technology-based early detection system. This plant’s early detection technology is beneficial for lowering farmers’ losses. The detection system based on digital images is also cost-effective. This paper aims to model the classification of corn plant diseases and pests based on digital images by implementing fuzzy discretization. Discretization is an essential technique to improve the knowledge extraction process of continuous-type data. It is also essential in some methods where continuous data must be processed or handled. Fuzzy discretization allows classes to have overlapping intervals so that they can handle information that is vague or unclear. We developed hypotheses and proved that different combinations of membership functions in fuzzy discretization affect classification performance. Empirical assessment using Monte Carlo resampling was carried out to obtain the generalizability of the performance of the best classification model of all proposed models. The best model is determined based on the number of metrics with the highest value and the highest metric on the Fscore and Kappa, a multiclass measure. The combination of digital image data preprocessing and classification methods also affects the performance of the classification model. We hope this work can provide an overview for experts in building early detection systems of corn plant diseases and pests using classification models based on fuzzy discretization.

Influence of Hyperparameters in Deep Learning Models for Coffee Rust Detection

Most of the world’s crops can be attacked by various diseases or pests, affecting their quality and productivity. In recent years, transfer learning with deep learning (DL) models has been used to detect diseases in maize, tomato, rice, and other crops. In the specific case of coffee, some recent works have used fixed hyperparameters to fine-tune the pre-trained models with the new dataset and/or applied data augmentation, such as image patching, to improve classifier performance. However, a detailed evaluation of the impact of architecture (e.g., backbone) and training (e.g., optimizer and learning rate) hyperparameters on the performance of coffee rust classification models has not been performed. Therefore, this paper presents a comprehensive study of the impact of five types of hyperparameters on the performance of coffee rust classification models. Specifically, eight pre-trained models are compared, each with four different amounts of transferred layers and three different numbers of neurons in the fully-connected (FC) layer, and the models are fine-tuned with three types of optimizers, each with three learning rate values. Comparing more than 800 models in terms of F1-score and accuracy, it is identified that the type of backbone is the hyperparameter with the greatest impact (with differences between models of up to 70%), followed by the optimizer (with differences of up to 20%). At the end of the study, specific recommendations are made on the values of the most suitable hyperparameters for the identification of this type of disease in coffee crops.

Federated Learning with Research Prototypes: Application to Multi-Center MRI-based Detection of Prostate Cancer with Diverse Histopathology.

Early prostate cancer detection and staging from MRI is extremely challenging for both radiologists and deep learning algorithms, but the potential to learn from large and diverse datasets remains a promising avenue to increase their performance within and across institutions. To enable this for prototype-stage algorithms, where the majority of existing research remains, we introduce a flexible federated learning framework for cross-site training, validation, and evaluation of custom deep learning prostate cancer detection algorithms. We introduce an abstraction of prostate cancer groundtruth that represents diverse annotation and histopathology data. We maximize use of this groundtruth if and when they are available using UCNet, a custom 3D UNet that enables simultaneous supervision of pixel-wise, region-wise, and gland-wise classification. We leverage these modules to perform cross-site federated training using 1400+ heterogeneous multi-parameteric prostate MRI exams from two University hospitals. We observe a positive result, with significant improvements in cross-site generalization performance with negligible intra-site performance degradation for both lesion segmentation and per-lesion binary classification of clinically-significant prostate cancer. Cross-site lesion segmentation performance intersection-over-union (IoU) improved by 100%, while cross-site lesion classification performance overall accuracy improved by 9.5-14.8%, depending on the optimal checkpoint selected by each site. Federated learning can improve the generalization performance of prostate cancer detection models across institutions while protecting patient health information and institution-specific code and data. However, even more data and participating institutions are likely required to improve the absolute performance of prostate cancer classification models. To enable adoption of federated learning with limited re-engineering of federated components, we open-source our FLtools system at https://federated.ucsf.edu, including examples that can be easily adapted to other medical imaging deep learning projects.

Research of Convolutional Neural Network on Image Classification

With the progress of artificial intelligence, technology based on deep learning is becoming more and more mature, and the application of deep convolutional neural network for image classification has become a popular topic for researchers. The number of the structure of deep convolutional neural network for image classification is keep increasing, and its performance is consistently improving, gradually replace that of traditional methods. According to the process of model development and model optimization, this paper divides the convolutional neural network into two models: classical deep convolutional neural network model and attention mechanism deep convolutional neural network model. The construction methods and characteristics of various kinds of deep convolutional neural network models are comprehensively reviewed, and the performance of various classification models is compared and analyzed. Finally, the problems of deep convolutional neural network model are presented.

Sustaining the High Performance of AI-Based Network Traffic Classification Models

Network traffic classification plays an essential role in network measurement and management. Emerging Artificial Intelligence (AI) algorithms have become a viable solution to encrypted network traffic classification. Nonetheless, the classification performance of existing AI-based traffic classifiers is restricted to a limited number of network applications depending on the coverage of the knowledge database. Such AI-based traffic classifiers cannot maintain high performance to provide accurate traffic classification when dealing with updated or new network applications. To tackle the issues, we present an autonomous model update mechanism to sustain the high performance of AI-based traffic classifiers. Specifically, an instability check algorithm is derived to evaluate if the current classifier requires an update. A filtering algorithm is proposed to extract unknown traffic and build a new knowledge database based on a new metric, i.e., familiarity, defined based on the prediction confidence and instability. Extensive experiment results demonstrate that our proposed updating mechanism can provide prompt model updates and establish a proper new knowledge base to maintain high accuracy in various experimental scenarios. Moreover, the comparison is conducted and the results show the proposed familiarity-based filtering algorithm can filter about 7 and 3 times more true positive packets in the two considered scenarios, respectively.

Interpretable machine learning assisted spectroscopy for fast characterization of biomass and waste.

The combination of machine learning and infrared spectroscopy was reported as effective for fast characterization of biomass and waste (BW). However, this characterization process is lack of interpretability towards its chemical insights, leading to less satisfactory recognition for its reliability. Accordingly, this paper aimed to explore the chemical insights of the machine learning models in the fast characterization process. A novel dimensional reduction method with significant physicochemical meanings was thus proposed, where the high loading spectral peaks of BW were selected as input features. Combined with functional groups attribution of these spectral peaks, the machine learning models established based on the dimensionally reduced spectral data could be explained with clear chemical insights. The performance of classification and regression models between the proposed dimensional reduction method and principal component analysis method was compared. The influence mechanism of each functional group on the characterization results were discussed. CH deformation, CC stretch & CO stretch and ketone/aldehyde CO stretch played essential roles in C, H/ LHV and O prediction, respectively. The results of this work demonstrated the theoretical fundamentals of the machine learning and spectroscopy based BW fast characterization method.

Multi-Layer Perceptron For Diagnosing Stroke With The SMOTE Method In Overcoming Data Imbalances

Stroke is the sudden loss of brain function due to an interruption of the blood supply to the brain. Stroke is a dangerous disease that can even cause death for patients. The diagnosis of stroke must be made quickly and precisely to increase the likelihood that the patient can live a normal life again. In making a diagnosis, several factors can influence the patient to get a stroke diagnosis, including symptoms of hypertension to heart disease. From these problems, the researcher wants to classify the diagnosis of stroke so that stroke can get earlier treatment so that patients do not experience prolonged illness. The data used in this study is a stroke dataset with 4861 data labeled 0 which indicates no stroke, and 249 data labeled 1 which indicates a stroke diagnosis. This study uses the Synthetic Minority Over-sampling (SMOTE) method that will be applied to the Multi-Layer Perceptron algorithm so that researchers can get the performance of the stroke diagnosis classification model. Researchers use the SMOTE method so that the data in the classification model is balanced so that the model can make accurate predictions and avoid overfitting on the Multi-Layer Perceptron so that the accuracy in predicting stroke is better than just using an ordinary Multi-Layer Perceptron. The results of the confusion matrix analysis show that SMOTE can increase the prediction of stroke diagnosis from 12,5% to 84,89% in optimal test.

Automatic breast cancer diagnosis based on hybrid dimensionality reduction technique and ensemble classification.

Feature selection in the face of high-dimensional data can reduce overfitting and learning time, and at the same time improve the accuracy and efficiency of the system. Since there are many irrelevant and redundant features in breast cancer diagnosis, removing such features leads to more accurate prediction and reduced decision time when dealing with large-scale data. Meanwhile, ensemble classifiers are powerful techniques to improve the prediction performance of classification models, where several individual classifier models are combined to achieve higher accuracy. In this paper, an ensemble classifier algorithm based on multilayer perceptron neural network is proposed for the classification task, in which the parameters (e.g., number of hidden layers, number of neurons in each hidden layer, and weights of links) are adjusted based on an evolutionary approach. Meanwhile, this paper uses a hybrid dimensionality reduction technique based on principal component analysis and information gain to address this problem. The effectiveness of the proposed algorithm was evaluated based on the Wisconsin breast cancer database. In particular, the proposed algorithm provides an average of 17% better accuracy compared to the best results obtained from the existing state-of-the-art methods. Experimental results show that the proposed algorithm can be used as an intelligent medical assistant system for breast cancer diagnosis.

Performance Evaluation of Classification Algorithm for Movie Review Sentiment Analysis

The majority of the current research on sentiment analysis, which covers topics like political reviews, movie reviews, and product reviews, has developed quickly. The classification and clustering stage of sentiment analysis research involves a number of subjects. Some of them cover text classification comparison research and algorithm performance optimization. An intricate issue in sentiment analysis research is dealing with unstructured or semi-structured data. The sentiment analysis procedure and improving the efficacy of the classifier’s algorithm are both hampered by unstructured data. In order to manage unstructured data successfully and provide accurate and relevant information, unique strategies are required. The proposed classification model performance evaluation using Support Vector Machine, Naive Bayes, K-Nearest Neighbor, and Decision Tree is specifically covered in this paper. According to the study’s findings, SVM has an accuracy rate of 96% and Naive Bayes is 86%. While the decision tree’s gain accuracy is 78 percent and the kNN classification model’s gain accuracy is 78 percent, respectively. The test results demonstrate that SVM is superior to other classification models in terms of accuracy performance.

Fused deep neural networks for sustainable and computational management of heat-transfer pipeline diagnosis

We propose deep learning-based models for the risk detection of underground pipelines. To build effective diagnosis models, we construct two types of deep neural network frameworks. First, we propose an image segmentation model with two parallel encoder structures for analyzing RGB and thermal images to detect risk-suspected regions due to pipeline rupture. In addition, for pipeline risk-stage recognition, we compare the performance of various image classification models and select the optimal model. Then, by integrating the image segmentation and classification models, we propose a fused model that performs risk-suspected region detection and risk-level detection using a single inference. The proposed image segmentation model achieves IoU and Dice coefficient values of 0.8373 and 0.9024, respectively, indicating higher performance in suspected-region detection compared with other competing models and demonstrating robust performance against false recognition. In risk-level detection, we confirm that the DenseNet-based model has the highest performance among the classification models, scoring 94.10% and 95.65% in terms of heat and leak accuracy, respectively. Finally, we validate that the fusion model, which recorded 0.8160, 0.8711, 93.90%, and 95.05% in terms of IoU, Dice coefficient, heat accuracy, and leak accuracy, respectively, can process 30 frames of video per second in real-time at the cost of a slight drop in performance.

Multiclass Reinforced Active Learning for Droplet Pinch-Off Behaviors Identification in Inkjet Printing

Abstract Inkjet printing (IJP) is one of the promising additive manufacturing techniques that yield many innovations in electronic and biomedical products. In IJP, the products are fabricated by depositing droplets on substrates, and the quality of the products is highly affected by the droplet pinch-off behaviors. Therefore, identifying pinch-off behaviors of droplets is critical. However, annotating the pinch-off behaviors is burdensome since a large amount of images of pinch-off behaviors can be collected. Active learning (AL) is a machine learning technique which extracts human knowledge by iteratively acquiring human annotation and updating the classification model for the pinch-off behaviors identification. Consequently, a good classification performance can be achieved with limited labels. However, during the query process, the most informative instances (i.e., images) are varying and most query strategies in AL cannot handle these dynamics since they are handcrafted. Thus, this paper proposes a multiclass reinforced active learning (MCRAL) framework in which a query strategy is trained by reinforcement learning (RL). We designed a unique intrinsic reward signal to improve the classification model performance. Moreover, how to extract the features from images for pinch-off behavior identification is not trivial. Thus, we used a graph convolutional network for droplet image feature extraction. The results show that MCRAL excels AL and can reduce human efforts in pinch-off behavior identification. We further demonstrated that, by linking the process parameters to the predicted droplet pinch-off behaviors, the droplet pinch-off behavior can be adjusted based on MCRAL.

Supervised Relation Extraction Between Suicide-Related Entities and Drugs: Development and Usability Study of an Annotated PubMed Corpus.

Drug-induced suicide has been debated as a crucial issue in both clinical and public health research. Published research articles contain valuable data on the drugs associated with suicidal adverse events. An automated process that extracts such information and rapidly detects drugs related to suicide risk is essential but has not been well established. Moreover, few data sets are available for training and validating classification models on drug-induced suicide. This study aimed to build a corpus of drug-suicide relations containing annotated entities for drugs, suicidal adverse events, and their relations. To confirm the effectiveness of the drug-suicide relation corpus, we evaluated the performance of a relation classification model using the corpus in conjunction with various embeddings. We collected the abstracts and titles of research articles associated with drugs and suicide from PubMed and manually annotated them along with their relations at the sentence level (adverse drug events, treatment, suicide means, or miscellaneous). To reduce the manual annotation effort, we preliminarily selected sentences with a pretrained zero-shot classifier or sentences containing only drug and suicide keywords. We trained a relation classification model using various Bidirectional Encoder Representations from Transformer embeddings with the proposed corpus. We then compared the performances of the model with different Bidirectional Encoder Representations from Transformer-based embeddings and selected the most suitable embedding for our corpus. Our corpus comprised 11,894 sentences extracted from the titles and abstracts of the PubMed research articles. Each sentence was annotated with drug and suicide entities and the relationship between these 2 entities (adverse drug events, treatment, means, and miscellaneous). All of the tested relation classification models that were fine-tuned on the corpus accurately detected sentences of suicidal adverse events regardless of their pretrained type and data set properties. To our knowledge, this is the first and most extensive corpus of drug-suicide relations.

Enhancing Small Medical Dataset Classification Performance Using GAN

Developing an effective classification model in the medical field is challenging due to limited datasets. To address this issue, this study proposes using a generative adversarial network (GAN) as a data-augmentation technique. The research aims to enhance the classifier’s generalization performance, stability, and precision through the generation of synthetic data that closely resemble real data. We employed feature selection and applied five classification algorithms to thirteen benchmark medical datasets, augmented using the least-square GAN (LS-GAN). Evaluation of the generated samples using different ratios of augmented data showed that the support vector machine model outperforms other methods with larger samples. The proposed data augmentation approach using a GAN presents a promising solution for enhancing the performance of classification models in the healthcare field.

오류 라벨을 포함한 분류모형의 강건성 모의실험연구

In order to increase the accuracy and performance of classification models, the quality of data must be guaranteed. There are various factors that affect the quality of these data, such as measurement accuracy, missing value, and sampling biases. This study focuses on the robustness of the classification model for mislabeled data. The reason we are interested in mislabeled data is that it significantly degrades the performance of the model more than other common measurement errors. In this study, the efficiency of various classification methods for data with mislabels is reviewed. Fisher's linear discriminant analysis model (LDA), quadratic discriminant analysis model (QDA), and logistic regression model were considered in the statistical model-based method, and SVM (support vector machine) and boosting are compared in the algorithm-based method. As a result of simulation study, it is found that the statistical model based approaches show better results in terms of robustness in the data assuming various situations of mislabels.

CNN-Wavelet scattering textural feature fusion for classifying breast tissue in mammograms

Visual interpretation from radiologists employs computer-aided diagnosis (CAD) to make clinical diagnoses by analyzing breast tissue images and assessing their texture. Aside from needing more training images for Convolutional Neural Network (CNN), selecting textural feature input to the deep learning-based CAD is challenging to control since the entire feature map is extracted automatically and only considers spatial domain analysis. Hence, integrating spatial and frequency information from the images requires complex feature representation in a classification problem. Wavelet representation could improve the feature descriptors in acquiring information from the spatial-frequency analysis. However, wavelet transforms’ shift insensitivity may affect class feature representation by suppressing high-frequency information. This study uses CNN, and wavelet scattering (WS) features to classify fatty and fibroglandular tissue with benign and malignant masses from digital mammograms to overcome CNN overfitting based on a limited dataset. Scattered WS coefficients formed from layers of wavelet dilations and averaging are utilized to retain the loss of high-frequency signal from the images, while simultaneously, the added CNN features improve sparse image representation resulting from convoluted spatial information of the images. Finally, the model is cascaded with an ensemble classifier for classifying fatty and fibroglandular tissue and mass images, n = 112, from the INbreast mammogram dataset. The best performance of mass and breast tissue classification models reach 98.0 % and 99.3 % on 10-fold cross-validation accuracy. By combining textural features from WS and deep learning model architecture, the model may adopt a training system with fewer data without augmentation to achieve good performance and low complexity costs.

Near infrared spectroscopy for the identification of live anurans: Towards rapid and automated identification of species in the field

In megadiverse regions, such as the Amazon, the identification of species generally requires specialists that are often not available. Therefore, the use of new species-recognition tools is necessary to streamline surveys and avoid errors in species identification that lead to ineffective decision-making. Near infrared spectroscopy is a quick and non-destructive tool that has been widely used in the recognition of biodiversity. In addition to being used as an indicator group, anurans have species with high morphological diversity, which make them the focus of studies and application of new tools that help in the identification and recognition at the species level. In this study, the viability of recognition of species of live Amazonian frogs under field conditions using the near infrared technique and portable equipment was examined. The performance of classification models based on a linear discriminant analysis, built using spectra obtained from the dorsal and ventral surfaces of four pairs of phylogenetically-close and morphologically-similar species was evaluated. It was possible to distinguish the species of live anurans in five of the eight species studied with hit rates above 80% when using only one spectral reading per individual. The overall mean of correct prediction of the models was below that of previous studies that tested the method with anurans, which are likely to be due to particularities in the acquisition of spectra under field conditions and live species. Therefore, suggestions are made to improve the predictive capacity of the techniques.

E-Nose System Based on Fourier Series for Gases Identification and Concentration Estimation From Food Spoilage

This work presents an electronic nose (EN)-based gas identification and concentration estimation method for the detection of food spoilage. The response data of sensors were acquired through a commercial gas sensor array and data acquisition circuit board and transformed into pictures with the form of the Fourier series. A convolutional neural network (CNN) model was used to identify the pictures from the conversion of sensor data, thus achieving the purpose of identifying the gases (C2H5OH, NH3, and H2S). In order to solve the problem of sample imbalance and to improve the generalization performance of classification models, the synthetic minority oversampling technique (SMOTE) and dropout technique were employed. Fivefold cross-validation was used to evaluate the performance of the model, of which the gas identification accuracy rate reached 96.67%. Moreover, a gas concentration regression model with the advantages of simplicity and strong interpretability was further proposed to estimate the concentrations of C2H5OH, NH3, and H2S. The mean absolute errors and coefficients of determination for the concentration estimation of C2H5OH, NH3, and H2S are (3.71 ppm, 0.968), (0.50 ppm, 0.968), and (0.13 ppm, 0.99), respectively. Furthermore, we used our model to evaluate the freshness of kiwifruit, pork, and beef, and it showed satisfactory predictive performance. The method proposed in this work realizes high-precision detection of gases from food spoilage and has a good application prospect in the rapid judgment of food freshness on the EN system.

Active Weighted Aging Ensemble for drifted data stream classification

PurposeOne of the significant problems in data stream classification is the concept drift phenomenon, which consists of the change in probabilistic characteristics of the classification task. Such changes in posterior probability destabilize the classification model performance, seriously degrading its quality. It is necessary to design appropriate strategies to counteract this phenomenon, allowing the classifier to adapt to the changing probabilistic characteristics. It is tough to propose such an approach with limited access to data labels. A human bias of high quality is usually costly, so to minimize the expenses related to this process, it is also necessary to propose learning strategies based on semi-supervised learning. Such strategies employ active learning methods indicating which of the incoming objects are valuable to be labeled for improving the classifier's performance. MethodsThis paper proposes Active Weighted Aging Ensemble algorithm, a novel chunk-based method for non-stationary data stream classification. It employs a classifier ensemble approach and utilizes the changing ensemble lineup to react to concept drift appropriately. It also proposed a new active learning method, considering a limited budget that may be applied to any data stream classifier. ResultsAWAE has been evaluated through computer experiments using real and synthetic data streams, confirming the proposed algorithm's high quality over state-of-the-art methods. ConclusionThe research conducted on benchmark data streams confirmed the effectiveness of the proposed solution and highlighted its strengths in comparison with state-of-the-art methods. The estimated computational complexity is acceptable and comparable to the benchmark algorithms.

Intelligent Discrete Deep Learning Based Classification Methodology in Chemometrics.

In recent years, deep learning models have attracted much attention for classification purposes in chemometrics. The popularity of deep learning models in this field comes from their unique features like universal approximation capability with the desired accuracy. Deep learning classifiers use several intelligent processing layers to model mixed, complex, and nonlinear patterns in the underlying data sets, which is why the development of deep learning based models has never been stopped in the chemometrics literature. Despite the variety of deep learning classification models used in this field, they all use a continuous distance-based cost function in their learning processes. Although using a continuous cost function for learning deep classifiers is a common approach, it conflicts with the discrete nature of the classification problem. In fact, applying a continuous cost function for inherently discrete classification problems can reduce the performance of the classification. In this research, a novel discrete learning based classification approach is proposed and implemented on a deep feed-forward neural network as one of the most commonly used deep learning models to develop a different learning process for deep classification models. The basis of the proposed learning approach is maximizing a discrete matching function of the actual and fitted values instead of minimizing a continuous distance-based cost function. The proposed classification approach is evaluated on five benchmark data sets in the chemistry field. The empirical results indicated the superiority of the proposed discrete deep learning approach over its classic continuous form. The results of this study demonstrate the important effect of discrete learning processes on the performances of deep learning classification models. Therefore, the proposed methodology can be a powerful alternative to common classification approaches to analyze chemical data in the chemometrics field.