Classification Model Research Articles

Application of convolutional neural networks in image classification and applications of improved convolutional neural networks

Abstract. This paper reviews the application and improvement of convolutional neural networks (CNNs) in image classification. Firstly, a shallow CNN for interstitial lung disease image classification is presented. This model suppresses overfitting through a unique network architecture and optimisation algorithm. Next, the improved VGG16 architecture and MIDNet18 model are discussed and their superior performance in brain tumour image classification is demonstrated. Subsequently, a CNN-CapsNet model for cervical cancer image classification and its improvement are presented and the customised model is compared with the conventional VGG-16 CNN architecture in the paper. Next, the application of sparse convolutional kernels and hybrid sparse convolutional kernels (HDCs) in solving the problem of computational resource consumption is presented. Subsequently, methods for solving the problem of limited training data through transfer learning and network data augmentation techniques are discussed, as well as GAN-generated datasets for solving the overfitting problem. Finally, the effect of degraded images on the classification effectiveness of CNNs is explored. The results show that the improved CNN architecture and algorithms have significant effects in solving the problems of overfitting and computational resource consumption, and can significantly improve the accuracy and efficiency of image classification. And degraded images do adversely affect the accuracy of CNN for image classification.

An Artificial Intelligence Approach for Test-Free Identification of Sarcopenia.

The diagnosis of sarcopenia relies extensively on human and equipment resources and requires individuals to personally visit medical institutions. The objective of this study was to develop a test-free, self-assessable approach to identify sarcopenia by utilizing artificial intelligence techniques and representative real-world data. This multicentre study enrolled 11 661 middle-aged and older adults from a national survey initialized in 2011. Follow-up data from the baseline cohort collected in 2013 (n = 9403) and 2015 (n = 10 356) were used for validation. Sarcopenia was retrospectively diagnosed using the Asian Working Group for Sarcopenia 2019 framework. Baseline age, sex, height, weight and 20 functional capacity (FC)-related binary indices (activities of daily living = 6, instrumental activities of daily living = 5 and other FC indices = 9) were considered as predictors. Multiple machine learning (ML) models were trained and cross-validated using 70% of the baseline data to predict sarcopenia. The remaining 30% of the baseline data, along with two follow-up datasets (n = 9403 and n = 10 356, respectively), were used to assess model performance. The study included 5634 men and 6027 women (median age = 57.0 years). Sarcopenia was identified in 1288 (11.0%) individuals. Among the 20 FC indices, the running/jogging 1 km item showed the highest predictive value for sarcopenia (AUC [95%CI] = 0.633 [0.620-0.647]). From the various ML models assessed, a 24-variable gradient boosting classifier (GBC) model was selected. This GBC model demonstrated favourable performance in predicting sarcopenia in the holdout data (AUC [95%CI] = 0.831 [0.808-0.853], accuracy = 0.889, recall = 0.441, precision = 0.475, F1 score = 0.458, Kappa = 0.396 and Matthews correlation coefficient = 0.396). Further model validation on the temporal scale using two longitudinal datasets also demonstrated good performance (AUC [95%CI]: 0.833 [0.818-0.848] and 0.852 [0.840-0.865], respectively). The model's built-in feature importance ranking and the SHapley Additive exPlanations method revealed that lifting 5 kg and running/jogging 1 km were relatively important variables among the 20 FC items contributing to the model's predictive capacity, respectively. The calibration curve of the model indicated good agreement between predictions and actual observations (Hosmer and Lemeshow p = 0.501, 0.451 and 0.374 for the three test sets, respectively), and decision curve analysis supported its clinical usefulness. The model was implemented as an online web application and exported as a deployable binary file, allowing for flexible, individualized risk assessment. We developed an artificial intelligence model that can assist in the identification of sarcopenia, particularly in settings lacking the necessary resources for a comprehensive diagnosis. These findings offer potential for improving decision-making and facilitating the development of novel management strategies of sarcopenia.

A Data-Driven Model for Classification of Traditional Chinese Medicine Materials

This paper proposes a data-driven classification model for traditional Chinese medicinal herbs based on mid-infrared spectral data. Addressing the limitations of traditional identification methods when the herbs’ appearance is damaged or incomplete, this study employs machine learning techniques to achieve accurate classification of medicinal herb types through spectral data preprocessing and feature extraction. Firstly, the Savitzky-Golay convolution smoothing method and Standard Normal Variate (SNV) transformation were used for denoising the spectral data. Then, Principal Component Analysis (PCA) was employed to reduce the dimensionality of the high-dimensional spectral data and extract the key features. Finally, the Gaussian Mixture Model (GMM) was applied to cluster the reduced data, categorizing the medicinal herbs into six classes. The results show that this method produces the accurate and stable classification. The constructed model is not only applicable to the classification and origin identification of medicinal herbs but also provides an important reference value for the classification and origin identification of other plant species.

Machine learning-driven prediction of EBNA1 inhibitors against Epstein–Barr virus in nasopharyngeal carcinoma

Nasopharyngeal carcinoma (NPC), particularly prevalent in regions such as Malaysia, is a significant health concern often linked to Epstein-Barr virus (EBV) infection. The EBV nuclear antigen 1 (EBNA1), crucial for EBV survival and NPC tumorigenicity, has emerged as a potential therapeutic target for EBV-positive NPC. In this study, we utilized quantitative structure-activity relationship (QSAR) models to predict potential inhibitors of EBNA1. These models were developed based on the molecular fingerprints of known EBNA1 inhibitors, using both classification and regression approaches. Our QSAR classification models demonstrated consistently high precision, recall, F1 score, and accuracy scores across the training set. The top-performing models, constructed using logistic regression algorithms, achieved perfect precision scores of 1.000 in the test set evaluation. These models’ recall, F1 score, and accuracy scores were 0.571, 0.727, and 0.667, respectively. On the other hand, the best-performing model among the regression models was built using the sequential minimal optimization regression algorithm, achieving a correlation coefficient of 0.703. The mean absolute error and root mean square error of this QSAR regression model were 0.173 and 0.217, respectively, whereas the relative absolute error was 0.689. We screened the enamine advanced compound library using this regression model to predict compounds with potential EBNA1 inhibitory effects. This led to the identification of the top 10 compounds with the most promising predicted EBNA1 inhibitory properties.

Peptipedia v2.0: a peptide sequence database and user-friendly web platform. A major update.

In recent years, peptides have gained significant relevance due to their therapeutic properties. The surge in peptide production and synthesis has generated vast amounts of data, enabling the creation of comprehensive databases and information repositories. Advances in sequencing techniques and artificial intelligence have further accelerated the design of tailor-made peptides. However, leveraging these techniques requires versatile and continuously updated storage systems, along with tools that facilitate peptide research and the implementation of machine learning for predictive systems. This work introduces Peptipedia v2.0, one of the most comprehensive public repositories of peptides, supporting biotechnological research by simplifying peptide study and annotation. Peptipedia v2.0 has expanded its collection by over 45% with peptide sequences that have reported biological activities. The functional biological activity tree has been revised and enhanced, incorporating new categories such as cosmetic and dermatological activities, molecular binding, and antiageing properties. Utilizing protein language models and machine learning, more than 90 binary classification models have been trained, validated, and incorporated into Peptipedia v2.0. These models exhibit average sensitivities and specificities of 0.877±0.0530 and 0.873±0.054, respectively, facilitating the annotation of more than 3.6 million peptide sequences with unknown biological activities, also registered in Peptipedia v2.0. Additionally, Peptipedia v2.0 introduces description tools based on structural and ontological properties and user-friendly machine learning tools to facilitate the application of machine learning strategies to study peptide sequences. Database URL: https://peptipedia.cl/.

An explainable deep learning model to predict partial anomalous pulmonary venous connection for patients with atrial septal defect

BackgroundPatients with partial anomalous pulmonary venous connection (PAPVC) usually present asymptomatic and accompanied by intricate anatomical types, which results in missed diagnosis from atrial septal defect (ASD). The present study aimed to explore the predictive variables of PAPVC from patients with ASD and constructed an explainable prediction model based on deep learning.MethodsThe retrospective study included 834 inpatients with ASD in Women and Children's Hospital, Qingdao University from January 2018 to January 2023. They were separated into two groups based on the presence of PAPVC. Propensity score matching and SMOTE were used to balance the baseline data between groups. The differential variables between the two groups were determined by univariate logistic regression. The patients were randomly divided into the training set and the validation set in a ratio of 8:2. Support vector machines (SVM), Random forest, Decision tree, XGBoost, and LightGBM were used to build models by differential variables. The classification performance of models was compared. Split, gain and SHAP were used to measure the importance of differential variables and improve the interpretability of the model. Moreover, a portion of the patients was included in the validation set to test the performance of the selected models.ResultsThree hundred twenty-eight patients with ASD and patients with 82 PAPVC were included in the training set and the validation set, respectively. The selection of 10 differential variables was based on univariate logistic regression, including right atrial diameter (longitudinal axis and transverse axis), right ventricular diameter, left atrial diameter, left ventricular end-diastolic diameter, left ventricular end-systolic diameter, P-wave voltage, P-wave interval PR interval, and QRS-wave voltage. In the classification model established based on differential variables, the LightGBM model achieved the highest performance on the validation set (AUC = 0.93). Based on variables importance analysis, the LightGBM-Clinic model was retrained by P-wave voltage, P-wave interval, PR interval, QRS wave interval, and right ventricular diameter, and performed excellently (AUC = 0.90). The AUC of the LightGBM-Clinic model was 0.87 in the test set.ConclusionIn this study, the LightGBM model performs excellently in determining whether patients with ASD are accompanied by PAPVC. ECG parameters such as P-wave voltage were important to predictive value and enhance the explainability of the model.

Machine Learning Approaches for the Fusion of Near-Infrared, Mid-Infrared, and Raman Data to Identify Cartilage Degradation in Human Osteochondral Plugs.

Vibrational spectroscopy methods such as mid-infrared (MIR), near-infrared (NIR), and Raman spectroscopies have been shown to have great potential for in vivo biomedical applications, such as arthroscopic evaluation of joint injuries and degeneration. Considering that these techniques provide complementary chemical information, in this study, we hypothesized that combining the MIR, NIR, and Raman data from human osteochondral samples can improve the detection of cartilage degradation. This study evaluated 272 osteochondral samples from 18 human knee joins, comprising both healthy and damaged tissue according to the reference Osteoarthritis Research Society International grading system. We established the one-block and multi-block classification models using partial least squares discriminant analysis (PLSDA), random forest, and support vector machine (SVM) algorithms. Feature modeling by principal component analysis was tested for the SVM (PCA-SVM) models. The best one-block models were built using MIR and Raman data, discriminating healthy cartilage from damaged with an accuracy of 77.5% for MIR and 77.8% for Raman using the PCA-SVM algorithm, whereas the NIR data did not perform as well achieving only 68.5% accuracy for the best model using PCA-SVM. The multi-block approach allowed an improvement with an accuracy of 81.4% for the best model by PCA-SVM. Fusing three blocks using MIR, NIR, and Raman by multi-block PLSDA significantly improved the performance of the single-block models to 79.1% correct classification. The significance was proven by statistical testing using analysis of variance. Thus, the study suggests the potential and the complementary value of the fusion of different spectroscopic techniques and provides valuable data analysis tools for the diagnostics of cartilage health.

Investigating the Fairness of Deep Learning Models in Breast Cancer Diagnosis Based on Race and Ethnicity

Breast cancer is the leading cancer affecting women globally. Despite deep learning models making significant strides in diagnosing and treating this disease, ensuring fair outcomes across diverse populations presents a challenge, particularly when certain demographic groups are underrepresented in training datasets. Addressing the fairness of AI models across varied demographic backgrounds is crucial. This study analyzes demographic representation within the publicly accessible Emory Breast Imaging Dataset (EMBED), which includes de-identified mammography and clinical data. We spotlight the data disparities among racial and ethnic groups and assess the biases in mammography image classification models trained on this dataset, specifically ResNet-50 and Swin Transformer V2. Our evaluation of classification accuracies across these groups reveals significant variations in model performance, highlighting concerns regarding the fairness of AI diagnostic tools. This paper emphasizes the imperative need for fairness in AI and suggests directions for future research aimed at increasing the inclusiveness and dependability of these technologies in healthcare settings. Code is available at: https://github.com/kuanhuang0624/EMBEDFairModels.

A Black-Box Watermarking Modulation for Object Detection Models

Training a Deep Neural Network (DNN) from scratch comes with a substantial cost in terms of money, energy, data, and hardware. When such models are misused or redistributed without authorisation, the owner faces significant financial and intellectual property (IP) losses. Therefore, there is a pressing need to protect the IP of Machine Learning models to avoid these issues. ML watermarking emerges as a promising solution for model traceability. Watermarking has been well-studied for image classification models, but there is a significant research gap in its application to other tasks like object detection, for which no effective methods have been proposed yet. In this paper, we introduce a novel black-box watermarking method for object detection models. Our contributions include a watermarking technique that maps visual information to text semantics and a comparative study of fine-tuning techniques’ impact on watermark detectability. We present the model’s detection performance and evaluate fine-tuning strategies’ effectiveness in preserving watermark integrity.

Enhancing Image Classification with Super-Resolution Techniques: A Comparative Study of SRResNet and SRGAN

Abstract. This paper explores the use of super-resolution (SR) approaches to enhance the performance of image classification, with particular attention to the effects of various SR models on classification accuracy. Using the ImageNet Dogs dataset for assessment, the paper examines two SR techniques: Super Resolution Generative Adversarial Networks (SRGAN) and Super Resolution Residual Networks (SRResNet). The research demonstrates that deep learning-based SR methods can enhance classification accuracy. Notably, SRResNet is identified as the superior method for improving classification performance, despite generating less visually appealing images compared to SRGAN. These finding highlights that while Generative Adversarial Networks (GANs) and perceptual loss functions can enhance image quality, their impact on classification accuracy may not always be substantial. The study offers important new perspectives on the relative merits of different SR approaches, highlighting the necessity of choosing the right SR methods in accordance with the particular demands of picture classification tasks. The results suggest that SRResNet, with its focus on accuracy over visual appeal, is more effective for boosting classification model performance, offering guidance for optimizing SR methods in practical image classification applications.

Machine Learning Models Based on [18F]FDG PET Radiomics for Bone Marrow Assessment in Non-Hodgkin Lymphoma

Non-Hodgkin lymphoma is a heterogeneous group of cancers that triggers bone marrow infiltration in 20–40% of cases. Bone marrow biopsy in combination with a visual assessment of [18F]FDG PET/CT images is used to assess the marrow status. Despite the potential of both techniques, they still have limitations due to the subjectivity of visual assessment. The present study aims to develop models based on bone marrow uptake in [18F]FDG PET/CT images at the time of diagnosis to differentiate bone marrow status. For this purpose, a model trained for skeleton segmentation and based on the U-Net architecture is retrained for bone marrow segmentation from CT images. The mask obtained from this segmentation together with the [18F]FDG PET image is used to extract radiomics features with which 11 machine learning models for marrow status differentiation are trained. The segmentation model yields very satisfactory results with Jaccard and Dice index values of 0.933 and 0.964, respectively. As for the classification models, a maximum F1_score_weighted and F1_score_macro of 0.962 and 0.747, respectively, are achieved. This highlights the potential of these features for bone marrow assessment, laying the foundation for a new clinical decision support system.

Classification of hand movements from EEG using a FusionNet based LSTM network.


Accurate classification of electroencephalogram (EEG) signals is crucial for advancing brain-computer interface (BCI) technology. However, current methods face significant challenges in classifying hand movement EEG signals, including effective spatial feature extraction, capturing temporal dependencies, and representing underlying signal dynamics.
Approach.
This paper introduces a novel multi-model fusion approach, FusionNet-LSTM, designed to address these issues. Specifically, it integrates Convolutional Neural Networks (CNN) for spatial feature extraction, Gated Recurrent Units (GRU) and Long Short-Term Memory (LSTM) networks for capturing temporal dependencies, and Autoregressive (AR) models for representing signal dynamics.
Main results.
Compared to single models and state-of-the-art methods, this fusion approach demonstrates substantial improvements in classification accuracy. Experimental results show that the proposed model achieves an accuracy of 87.1% in cross-subject data classification and 99.1% in within-subject data classification. Additionally, Gradient Boosting Trees were employed to evaluate the significance of various EEG features to the model.
Significance.
This study highlights the advantages of integrating multiple models and introduces a superior classification model, which is pivotal for the advancement of BCI systems.

Predictive Modelling of Crop Rotation Using Data Mining Approaches

Agriculture is crucial for economic growth and food security, particularly in agro-based countries. As the global population grows, the demand for food increases, necessitating improvements in agricultural productivity. Traditional methods have often fallen short, and innovative approaches such as data mining and machine learning are needed. This research aims to develop a predictive model for crop rotation using machine learning techniques. A comprehensive dataset was collected and preprocessed to train various algorithms. The proposed model demonstrated that machine learning could effectively predict suitable crops for cultivation, thereby enhancing crop yield and sustainability. The evaluation results were promising, with the Random Forest model achieving a precision of 0.67 to 1.00, recall of 0.43 to 1.00, and F1-score of 0.60 to 1.00; the Decision Tree model had a precision of 0.50 to 1.00, recall of 0.43 to 1.00, and F1-score of 0.50 to 1.00; and the K-Neighbors Classifier model showed precision of 0.40 to 1.00, recall of 0.43 to 1.00, and F1-score of 0.50 to 1.00.

A novel amalgamation of pre-processing technique and CNN model for accurate classification of power quality disturbances

A novel amalgamation of pre-processing technique and CNN model for accurate classification of power quality disturbances

Overview of Anomaly Detection Techniques across Different Domains: A Systematic Review

An anomaly, defined as something that deviates from what is normal, expected, or usual. It signifies abnormality or an irregularity that stands out from typical behaviours or patterns. Detecting anomalies is significant among numerous sectors due to the reasons of signal potential difficulties or opportunities. For an instance, in retail, detecting anomalies in sales data might prompt for further analysis into operational issues or customer behaviour to reduce losses and capitalize on its trends. Hence, different techniques are used for Anomaly Detection. However, anomaly detection using manual method are measured for time consuming, prone to error and can be tedious process. Therefore, different approaches have been considered for anomaly detection as AI (Artificial Intelligence) methods are efficient, faster, provides high level accuracy by effectively detecting the abnormalities. Owing to these aspects, this paper focuses on compiling different techniques and emphasizes on reviewing all anomaly detection using numerous techniques like ML (Machine Learning) and DL (Deep Learning) classifiers, statistical methods, one-class classification, clustering and density-based models which helps with identifying and comprehending the diversity of detection techniques that are applied in various domains like finance, retail, healthcare and cyber security. Various existing researches on anomaly detection are reviewed in the study. In addition to an overview, certain studies also deals with applications of detection models and future trends are reviewed in precise. Finally, the challenges are identified through the analysis of existing researchers and future recommendations are provided for overcoming the gaps that are intended to create promising work in this area.

An ensemble deep learning model for classification of students as weak and strong learners via multiparametric analysis

Academic data predictions are significantly important for improving the overall education system's effectiveness by providing early identification of weak students and personalized learning strategies. This paper proposes a deep learning model for the identification of weak and strong students using ensemble learning and multiparametric analysis. It combines several machine learning algorithms, including Naive Bayes, Support Vector Machines, Multi-Layer Perceptron, and Logistic Regression using an ensemble learning approach to enhance the model’s performance. Additionally, a custom 1D Convolutional Neural Network (CNN) is designed for classification. It utilizes multiparametric analysis to identify weak and strong students considering various parameters such as age, academic performance, location, and online learning behavior. The evaluation results indicate the performance of the proposed model has been improved in comparison to MLA FIS, SHMM, and DRL by 16.5%, 5.5%, and 2.4%, in terms of precision, 16.4%, 6.5%, and 3.5 % in terms of accuracy and 10.4%, 2.5% and 6.5% in terms of recall. These improvisations described that the model is efficient for multidomain feature extraction, ensemble classification, and high-variance feature selection, which result in a deeper understanding of student performance.

Enhancing mosquito classification through self-supervised learning

Traditional mosquito identification methods, relied on microscopic observation and morphological characteristics, often require significant expertise and experience, which can limit their effectiveness. This study introduces a self-supervised learning-based image classification model using the Bootstrap Your Own Latent (BYOL) algorithm, designed to enhance mosquito species identification efficiently. The BYOL algorithm offers a key advantage by eliminating the need for labeled data during pretraining, as it autonomously learns important features. During fine-tuning, the model requires only a small fraction of labeled data to achieve accurate results. Our approach demonstrates impressive performance, achieving over 96.77% accuracy in mosquito image analysis, with minimized both false positives and false negatives. Additionally, the model’s overall accuracy, measured by the area under the ROC curve, surpasses 99.55%, highlighting its robustness and reliability. A notable finding is that fine-tuning with just 10% of labeled data produces results comparable to using the full dataset. This is particularly valuable for resource-limited settings with limited access to advanced equipment and expertise. Our model provides a practical solution for mosquito identification, overcoming the challenges of traditional microscopic methods, such as the time-consuming process and reliance on specialized knowledge in healthcare services. Overall, this model supports personnel in resource-constrained environments by facilitating mosquito vector density analysis and paving the way for future mosquito species identification methodologies.

Artificial intelligence correctly classifies developmental stages of monarch caterpillars enabling better conservation through the use of community science photographs

Rapid technological advances and growing participation from amateur naturalists have made countless images of insects in their natural habitats available on global web portals. Despite advances in automated species identification, traits like developmental stage or health remain underexplored or manually annotated, with limited focus on automating these features. As a proof-of-concept, we developed a computer vision model utilizing the YOLOv5 algorithm to accurately detect monarch butterfly caterpillars in photographs and classify them into their five developmental stages (instars). The training data were obtained from the iNaturalist portal, and the photographs were first classified and annotated by experts to allow supervised training of models. Our best trained model demonstrates excellent performance on object detection, achieving a mean average precision score of 95% across all five instars. In terms of classification, the YOLOv5l version yielded the best performance, reaching 87% instar classification accuracy for all classes in the test set. Our approach and model show promise in developing detection and classification models for developmental stages for insects, a resource that can be used for large-scale mechanistic studies. These photos hold valuable untapped information, and we’ve released our annotated collection as an open dataset to support replication and expansion of our methods.

An optimized model for network intrusion detection in the network operating system environment

With the heavy reliance on computers and information technology to send and receive data across networks of various types, there has been concern about securing that data from intrusions and cyber-attacks. The expansion of network usage has led to an increase in hacker attacks, which has led to prioritizing cybersecurity precautions in detecting potential threats. Intrusion detection techniques are a critical security measure to protect networks in both personal and corporate environments that are managed by network operating systems. For this, the paper relies on designing a network intrusion detection model. Since deep neural networks (DNNs) are classic deep learning models known for their strong classification performance, making them popular in intrusion detection along with other machine learning algorithms, they have been chosen to improve intrusion classification models based on datasets for intrusion detection systems. The basic structure of this proposal is to adopt one of the optimization algorithms in extracting features from the dataset to obtain more accurate results in the classification and intrusion detection stage. The developed Corona Virus algorithm is adopted to improve the system performance by identifying optimal features. This algorithm, which consists of several stages, optimally selects individuals based on features from the NSL-KDD dataset used for intrusion detection. The resulting optimization solution acts as a network structure for the intrusion classification model based on machine learning and deep learning algorithms. The test results showed exceptional performance on the NSL-KDD dataset, where the proposed Convolution Neural Network CNN model achieved 99.3% accuracy for multi-class classification, while the Decision Tree (DT) achieved 88.64% accuracy for anomaly detection in bi-class classification.

Analysis of K-NN with the Integration of Bag of Words, TF-IDF, and N-Grams for Hate Speech Classification on Twitter

Social media has emerged as one of the primary communication channels in the modern world, but it has simultaneously become a platform where hate speech can spread easily. This study attempts to evaluate the performance of a hate speech classification model using the K-Nearest Neighbors (K-NN) algorithm along with various feature extraction techniques, specifically Bag of Words (BoW), Term Frequency-Inverse Document Frequency (TF-IDF), and N-Grams. The dataset used in this study consists of 13169 entries, which represent a diverse range of hate speech examples commonly encountered on social media platforms. In this experimental investigation, we assess the efficacy of the model using each feature extraction technique. The findings reveal that the K-NN model exhibits optimal performance when the k parameter is set to 3 (k=3). Under this configuration, the model achieves an accuracy of 86.88%, with a precision of 88.27%, a recall of 86.88%, and an F1-Score of 86.50%. These results show that the integration of TF-IDF feature extraction technique with K-NN algorithm produces superior performance in hate speech classification.