Multi-class Data Research Articles

Multi-Class Imbalanced Data Handling with Concept Drift in Fog Computing: A Taxonomy, Review, and Future Directions

A network of actual physical objects or “IoT components” linked to the internet and equipped with sensors, electronics, software, and network connectivity is known as the Internet of Things (IoT). This ability of the IoT components to gather and share data is made possible by this network connectivity. Many IoT devices are currently operating, which generate a lot of data. When these IoT devices started collecting data, the cloud was the only place to analyze, filter, pre-process, and aggregate it. However, when it comes to IoT, the cloud has restrictions regarding latency and a more centralized method of distributing programs. A new form of computing called Fog computing has been proposed to address the shortcomings of current cloud computing. In an IoT context, sensors regularly communicate signal information, and edge devices process the data obtained from these sensors using Fog computing. The sensors’ internal or external problems, security breaches, or the integration of heterogeneous equipment contribute to the imbalanced data, i.e., comparatively speaking, one class has more instances than the other. As a result of this data, the pattern extraction is imbalanced . Recent attempts have concentrated heavily on binary-class imbalanced concerns with exactly two classes. However, the classification of multi-class imbalanced data is an issue that needs to be fixed in Fog computing, even if it is widespread in other fields, including text categorization, human activity detection, and medical diagnosis. The study intends to deal with this problem. It presents a systematic, thorough, and in-depth comparative analysis of several binary-class and multi-class imbalanced data handling strategies for batch and streaming data in IoT networks and Fog computing. There are five major objectives in this study. First, reviewing the Fog computing concept. Second, outlining the optimization metric used in Fog computing. Third, focusing on binary and multi-class batch data handling for IoT networks and Fog computing. Fourth, reviewing and comparing the current imbalanced data handling methodologies for multi-class data streams. Fifth, explaining how to cope with the concept drift, including novel and recurring classes, targeted optimization measures, and evaluation tools. Finally, the best performance metrics and tools for concept drift, binary-class (batch and stream) data, and multi-class (batch and stream) data are highlighted.

Deep learning based capsule networks for breast cancer classification using ultrasound images

Purposes: Breast cancer (BC) is a disease in which the breast cells multiply uncontrolled. Breast cancer is one of the most often diagnosed malignancies in women worldwide. Early identification of breast cancer is critical for limiting the impact on affected people's health conditions. The influence of technology and artificial intelligence approaches (AI) in the health industry is tremendous as technology advances. Deep learning (DL) techniques are used in this study to classify breast lumps. Materials and Methods: The study makes use of two distinct breast ultrasound images (BUSI) with binary and multiclass classification. To assist the models in understanding the data, the datasets are exposed to numerous preprocessing and hyperparameter approaches. With data imbalance being a key difficulty in health analysis, due to the likelihood of not having a condition exceeding that of having the disease, this study applies a cutoff stage to impact the decision threshold in the datasets data augmentation procedures. The capsule neural network (CapsNet), Gabor capsule network (GCN), and convolutional neural network (CNN) are the DL models used to train the various datasets. Results: The findings showed that the CapsNet earned the maximum accuracy value of 93.62% while training the multiclass data, while the GCN achieved the highest model accuracy of 97.08\% when training the binary data. The models were also evaluated using a variety of performance assessment parameters, which yielded consistent results across all datasets. Conclusion: The study provides a non-invasive approach to detect breast cancer; and enables stakeholders, medical practitioners, and health research enthusiasts a fresh view into the analysis of breast cancer detection with DL techniques to make educated judgements.

An online ensemble classification algorithm for multi-class imbalanced data stream

An online ensemble classification algorithm for multi-class imbalanced data stream

A Comparison of Enhanced Ensemble Learning Techniques for Internet of Things Network Attack Detection

Over the past few decades, the Internet of Things (IoT) has become increasingly significant due to its capacity to enable low-cost device and sensor communication. Implementation has opened up many new opportunities in terms of efficiency, productivity, convenience, and security. However, it has also brought about new privacy and data security challenges, interoperability, and network reliability. The research issue is that IoT devices are frequently open to attacks. Certain machine learning (ML) algorithms still struggle to handle imbalanced data and have weak generalization skills when compared to ensemble learning. The research aims to develop security for IoT networks based on enhanced ensemble learning by using Grid Search and Random Search techniques. The method used is the ensemble learning approach, which consists of Random Forest (RF), Adaptive Boosting (AdaBoost), Gradient Boosting Machine (GBM), and Extreme Gradient Boosting (XGBoost). This study uses the UNSW-NB15 IoT dataset. The study's findings demonstrate that XGBoost performs better than other methods at identifying IoT network attacks. By employing Grid Search and Random Search optimization, XGBoost achieves an accuracy rate of 98.56% in binary model measurements and 97.47% on multi-class data. The findings underscore the efficacy of XGBoost in bolstering security within IoT networks.

Multi-Class Imbalanced Data Classification: A Systematic Mapping Study

Multi-Class Imbalanced Data Classification: A Systematic Mapping Study

ENSEMBLE CNN WITH ADASYN FOR MULTICLASS CLASSIFICATION ON CABBAGE PESTS

Image classification is a complex process influenced by various factors, one of which is the amount of image data. In the context of cabbage pest classification, data often exhibits a significant class imbalance, where certain pests are more prevalent than others. This imbalance can pose challenges during model training and evaluation, potentially leading to biases in favor of the majority pests and reduced accuracy in identifying and classifying the less common ones. This research aims to enhance the classification performance for multiclass data specific to cabbage pests. We propose an ensemble learning approach that combines Convolutional Neural Network (CNN), Support Vector Machine (SVM), and Bagging methods. To address the imbalance issue inherent in cabbage pest data, we employ the Adaptive Synthetic Sampling (ADASYN) resampling technique. The CNN acts as the primary image identifier and classifier for various cabbage pests. Subsequently, the CNN model is integrated into SVM and Bagging models to mitigate the challenges of imbalanced data in pest classification. The research outcomes demonstrate that our ensemble approach, in conjunction with the ADASYN resampling technique, achieves an impressive accuracy rate of 97%, signifying its potential for improved cabbage pest detection and classification.

Fragmentomics features of ovarian cancer.

Ovarian cancer (OC) is a major cause of cancer mortality in women worldwide. Due to the occult onset of OC, its nonspecific clinical symptoms in the early phase, and a lack of effective early diagnostic tools, most OC patients are diagnosed at an advanced stage. In this study, shallow whole-genome sequencing was utilized to characterize fragmentomics features of circulating tumor DNA (ctDNA) in OC patients. By applying a machine learning model, multiclass fragmentomics data achieved a mean area under the curve (AUC) of 0.97 (95% CI 0.962-0.976) for diagnosing OC. OC scores derived from this model strongly correlated with the disease stage. Further comparative analysis of OC scores illustrated that the fragmentomics-based technology provided additional clinical benefits over the traditional serum biomarkers cancer antigen 125 (CA125) and the Risk of Ovarian Malignancy Algorithm (ROMA) index. In conclusion, fragmentomics features in ctDNA are potential biomarkers for the accurate diagnosis of OC.

CVG-Net: novel transfer learning based deep features for diagnosis of brain tumors using MRI scans

Brain tumors present a significant medical challenge, demanding accurate and timely diagnosis for effective treatment planning. These tumors disrupt normal brain functions in various ways, giving rise to a broad spectrum of physical, cognitive, and emotional challenges. The daily increase in mortality rates attributed to brain tumors underscores the urgency of this issue. In recent years, advanced medical imaging techniques, particularly magnetic resonance imaging (MRI), have emerged as indispensable tools for diagnosing brain tumors. Brain MRI scans provide high-resolution, non-invasive visualization of brain structures, facilitating the precise detection of abnormalities such as tumors. This study aims to propose an effective neural network approach for the timely diagnosis of brain tumors. Our experiments utilized a multi-class MRI image dataset comprising 21,672 images related to glioma tumors, meningioma tumors, and pituitary tumors. We introduced a novel neural network-based feature engineering approach, combining 2D convolutional neural network (2DCNN) and VGG16. The resulting 2DCNN-VGG16 network (CVG-Net) extracted spatial features from MRI images using 2DCNN and VGG16 without human intervention. The newly created hybrid feature set is then input into machine learning models to diagnose brain tumors. We have balanced the multi-class MRI image features data using the Synthetic Minority Over-sampling Technique (SMOTE) approach. Extensive research experiments demonstrate that utilizing the proposed CVG-Net, the k-neighbors classifier outperformed state-of-the-art studies with a k-fold accuracy performance score of 0.96. We also applied hyperparameter tuning to enhance performance for multi-class brain tumor diagnosis. Our novel proposed approach has the potential to revolutionize early brain tumor diagnosis, providing medical professionals with a cost-effective and timely diagnostic mechanism.

Classification performance assessment for imbalanced multiclass data

The evaluation of diagnostic systems is pivotal for ensuring the deployment of high-quality solutions, especially given the pronounced context-sensitivity of certain systems, particularly in fields such as biomedicine. Of notable importance are predictive models where the target variable can encompass multiple values (multiclass), especially when these classes exhibit substantial frequency disparities (imbalance). In this study, we introduce the Imbalanced Multiclass Classification Performance (IMCP) curve, specifically designed for multiclass datasets (unlike the ROC curve), and characterized by its resilience to class distribution variations (in contrast to accuracy or Fβ\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_\\beta$$\\end{document}-score). Moreover, the IMCP curve facilitates individual performance assessment for each class within the diagnostic system, shedding light on the confidence associated with each prediction—an aspect of particular significance in medical diagnosis. Empirical experiments conducted with real-world data in a multiclass context (involving 35 types of tumors) featuring a high level of imbalance demonstrate that both the IMCP curve and the area under the IMCP curve serve as excellent indicators of classification quality.

A self-supervised leak detection method for natural gas gathering pipelines considering unlabeled multi-class non-leak data

Detecting leaks in natural gas gathering pipelines is paramount for the safe and reliable operation of the gas and oil industry. Due to the lack of leak data and the changes in leak features, semi-supervised leak detection methods that use normal data for health model learning have attracted much attention. However, these approaches usually consider one-class normal samples as health data, which may fail to fit the reality of unlabeled multi-class non-leak data under variable operating conditions. In addition, existing semi-supervised methods often suffer from insufficient representation learning as they employ step-by-step training or rely on the low-level reconstruction of autoencoders. To address the above two key challenges, this paper proposes a novel end-to-end self-supervised leak detection method, self-supervised multi-sphere support vector data description. Specifically, it utilizes the presented multi-sphere support vector data description to model unlabeled multi-class non-leak data and the introduced self-supervised learning strategy to boost the representation learning of the end-to-end semi-supervised model. Moreover, the categories of unlabeled multi-class non-leak data are learned in an unsupervised way through alternating feature clustering and pseudo-label-based classification. A robust leak score calculation method is also designed to improve the performance of the proposed method. Finally, the experimental results on the field data collected from pipelines demonstrate the effectiveness of the proposed method.

Industrial Big Data‐Driven Modeling and Prediction for Hot‐Rolled Strip Crown with Multigrade and Multispecification Data

In the field of hot rolling big data, the presence of different steel types, specifications, and data heterogeneity poses significant challenges to the accuracy and stability of using single machine learning regression technology for prediction. Therefore, this study proposes a hot‐rolled strip crown prediction method that combines data clustering and fusion modeling. First, this article introduces a relevant mechanism for designing cluster strategies. The optimal clustering strategy is determined through comparative experiments using rolling process parameters, strip size, and main material components as the clustering features. Subsequently, the K‐Means++ algorithm is used to effectively cluster the training and testing datasets based on this strategy, generating multiple clusters for both datasets. Finally, this study establishes seven different training models to match the most suitable regression prediction model for each cluster, and matching between each cluster and the model is determined through rigorous testing. The evaluation of the fusion model shows an R2 value of 0.829 and a root mean square error value of 3.974. The experimental results show that the proposed method outperforms traditional methods in solving the challenges of multiclass classification and data heterogeneity, providing strong data support for the intelligent control of the hot‐rolled strip crown in the future.

Dynamic Classification Ensembles for Handling Imbalanced Multiclass Drifted Data Streams

Machine learning models often encounter significant difficulties when dealing with multiclass imbalanced data streams in nonstationary environments. These challenges can lead to biased and unreliable predictions, which ultimately impact the overall performance of the models. To address these issues, we propose an innovative approach that integrates dynamic ensemble selection, an adaptive technique for managing imbalanced multiclass data streams, with a concept drift detector for recognizing stream changes and the K-nearest neighbor (KNN) algorithm to tackle issues related to class overlap. The primary objective was to improve the classification of imbalanced multiclass drifted data streams. The adaptive oversampling method generates synthetic samples to mitigate the issues associated with imbalanced data streams. This method utilizes KNN to ensure that the generated samples do not overlap. To handle incoming data streams, a drift detector assists in deciding whether to retain the existing classifiers or create a new one. Dynamic Ensemble Selection (DES) was utilized to select the most appropriate classifier for incoming data, aiming to optimize the performance of the classification task. The proposed method offers an effective solution for achieving an accurate and resilient classification in the context of imbalanced multiclass drifted data streams. To evaluate the effectiveness of our proposal, we conducted experiments on a variety of datasets, including benchmark datasets, real application stream datasets, and synthetic data streams. The experimental results demonstrate the superiority of our contribution in addressing the challenges posed by imbalanced multiclass drifted data streams.

Smooth Information Criterion for Regularized Estimation of Item Response Models

Item response theory (IRT) models are frequently used to analyze multivariate categorical data from questionnaires or cognitive test data. In order to reduce the model complexity in item response models, regularized estimation is now widely applied, adding a nondifferentiable penalty function like the LASSO or the SCAD penalty to the log-likelihood function in the optimization function. In most applications, regularized estimation repeatedly estimates the IRT model on a grid of regularization parameters λ. The final model is selected for the parameter that minimizes the Akaike or Bayesian information criterion (AIC or BIC). In recent work, it has been proposed to directly minimize a smooth approximation of the AIC or the BIC for regularized estimation. This approach circumvents the repeated estimation of the IRT model. To this end, the computation time is substantially reduced. The adequacy of the new approach is demonstrated by three simulation studies focusing on regularized estimation for IRT models with differential item functioning, multidimensional IRT models with cross-loadings, and the mixed Rasch/two-parameter logistic IRT model. It was found from the simulation studies that the computationally less demanding direct optimization based on the smooth variants of AIC and BIC had comparable or improved performance compared to the ordinarily employed repeated regularized estimation based on AIC or BIC.

A Classification Algorithm Inspired by the Chromatographic Separation Mechanism Dedicated to the Classification of Variable-length and Multi-class Vectors

Nowadays, one of the critical problems related to data mining is the processing of large data sets. This article presents an algorithm that may apply to the issues associated with classifying large-volume data sets. The motivation behind defining this type of algorithm was that the methods used to process this data type are subject to several significant limitations. The first considerable limitation of using classical classification methods is ensuring a constant data size. The second type of constraint is related to the data dimension. The last limitation in using classic classification algorithms is associated with the situation in which a given input vector may contain data belonging to many classes simultaneously, in which case we are talking about so-called multi-class vectors. The presented algorithm is inspired by the method of chromatographic separation of chemical substances. This method is widely and successfully used in analytical chemistry. As we know, in the case of chromatographic separation, we are dealing with a similar class of problems that occur when processing large data sets, firstly: the molecules of a chemical substance have a different number of molecules - i.e., they have different lengths, which corresponds to the situation that occurs when processing large data sets. In this work, a classification algorithm inspired by the mechanism of resolution chromatography is presented. The article presents the results of calculations for sample data sets. It discusses issues related to the properties of the defined algorithm, which concern the algorithm training process and the classification of single-class and multi-class data.

Supervised Clustering of Persian Handwritten Images Using Regularization and Dimension Reduction Methods

Clustering, as a fundamental exploratory data technique, not only is used to discover patterns and structures in complex datasets but also is utilized to group variables in high-dimensional data analysis. Dimension reduction through clustering helps identify important variables and reduce data dimensions without losing significant information. High-dimensional image datasets, such as Persian handwritten images, have numerous pixels, making statistical inference difficult. Such high-dimensionality property pose challenges for analysis and processing, requiring specialized techniques like clustering to extract information. Incorporating response variable information enhances clustering analysis, transforming it into a supervised method. This article evaluates a supervised clustering approach using Ridge and Lasso penalties, comparing them in analyzing a real dataset while identifying important variables. We demonstrate that despite choosing a small number of variables as important variables, Lasso penalty performs relatively well in predicting the labels of new observations for this multi-class dataset.

Topological learning in multiclass data sets.

We specialize techniques from topological data analysis to the problem of characterizing the topological complexity (as defined in the body of the paper) of a multiclass data set. As a by-product, a topological classifier is defined that uses an open subcovering of the data set. This subcovering can be used to construct a simplicial complex whose topological features (e.g., Betti numbers) provide information about the classification problem. We use these topological constructs to study the impact of topological complexity on learning in feedforward deep neural networks (DNNs). We hypothesize that topological complexity is negatively correlated with the ability of a fully connected feedforward deep neural network to learn to classify data correctly. We evaluate our topological classification algorithm on multiple constructed and open-source data sets. We also validate our hypothesis regarding the relationship between topological complexity and learning in DNN's on multiple data sets.

A Spectral Method for Identifiable Grade of Membership Analysis with Binary Responses.

Grade of membership (GoM) models are popular individual-level mixture models for multivariate categorical data. GoM allows each subject to have mixed memberships in multiple extreme latent profiles. Therefore, GoM models have a richer modeling capacity than latent class models that restrict each subject to belong to a single profile. The flexibility of GoM comes at the cost of more challenging identifiability and estimation problems. In this work, we propose a singular value decomposition (SVD)-based spectral approach to GoM analysis with multivariate binary responses. Our approach hinges on the observation that the expectation of the data matrix has a low-rank decomposition under a GoM model. For identifiability, we develop sufficient and almost necessary conditions for a notion of expectation identifiability. For estimation, we extract only a few leading singular vectors of the observed data matrix and exploit the simplex geometry of these vectors to estimate the mixed membership scores and other parameters. We also establish the consistency of our estimator in the double-asymptotic regime where both the number of subjects and the number of items grow to infinity. Our spectral method has a huge computational advantage over Bayesian or likelihood-based methods and is scalable to large-scale and high-dimensional data. Extensive simulation studies demonstrate the superior efficiency and accuracy of our method. We also illustrate our method by applying it to a personality test dataset.

FMRI-Based Multi-class DMDC Model Efficiently Decodes the Overlaps between ASD and ADHD.

Neurodevelopmental disorders comprise a group of neuropsychiatric conditions. Presently, behavior-based diagnostic approaches are utilized in clinical settings, but the overlapping features among these disorders obscure their recognition and management. Attention deficit hyperactivity disorder (ADHD) and autism spectrum disorder (ASD) have common characteristics across various levels, from genes to symptoms. Designing a computational framework based on the neuroimaging findings could provide a discriminative tool for ultimate more efficient treatment. Machine learning approaches, specifically classification methods are among the most applied techniques to reach this goal. We applied a novel two-level multi-class data maximum dispersion classifier (DMDC) algorithm to classify the functional neuroimaging data (utilizing datasets: ADHD-200 and autism brain imaging data exchange (ABIDE)) into two categories: Neurodevelopmental disorders (ASD and ADHD) or healthy participants, based on calculated functional connectivity values (statistical temporal correlation). Our model achieved a total accuracy of 62% for healthy controls. Specifically, it demonstrated an accuracy of 51% for healthy subjects, 61% for autism spectrum disorder, and 84% for ADHD. The support vector machine (SVM) model achieved an accuracy of 46% for both the healthy control and ASD groups, while the ADHD group classification accuracy was estimated to be 84%. These two models showed similar classification indices for the ADHD group. However, the discrimination power was higher in the ASD class. The method employed in this study demonstrated acceptable performance in classifying disorders and healthy conditions compared to the more commonly used SVM method. Notably, functional connections associated with the cerebellum showed discriminative power.

Online Machine Learning from Non-stationary Data Streams in the Presence of Concept Drift and Class Imbalance: A Systematic Review

In IoT environment applications generate continuous non-stationary data streams with in-built problems of concept drift and class imbalance which cause classifier performance degradation. The imbalanced data affects the classifier during concept detection and concept adaptation. In general, for concept detection, a separate mechanism is added in parallel with the classifier to detect the concept drift called a drift detector. For concept adaptation, the classifier updates itself or trains a new classifier to replace the older one. In case, the data stream faces a class imbalance issue, the classifier may not properly adapt to the latest concept. In this survey, we study how the existing work addresses the issues of class imbalance and concept drift while learning from nonstationarydata streams. We further highlight the limitation of existing work and challenges caused by other factors of class imbalance alongwith concept drift in data stream classification. Results of our survey found that, out of 1110 studies, by using our inclusion and exclusion criteria, we were able to narrow the pool of articles down to 35 that directly addressed our study objectives. The study found that issues such as multiple concept drift types, dynamic class imbalance ratio, and multi-class imbalance in presence of concept drift are still open for further research. We also observed that, while major research efforts have been dedicated to resolving concept drift and class imbalance, not much attention has been given to with-in-class imbalance, rear examples, and borderline instances when they exist with concept drift in multi-class data. This paper concludes with some suggested future directions.

Computer Vision-Aided Intelligent Monitoring of Coffee: Towards Sustainable Coffee Production

Earlier, researchers have employed computer vision-based automatic fruit detection to estimate coffee yield at the time of harvest, however, studies on the on-plant quantification of the coffee fruit are scarce. In this study, the latest version of the state-of-the-art algorithm YOLOv7 (You Only Look Once) was used for the first time. YOLOv7 was trained with 324 annotated images of fruit bearing coffee branches followed by its evaluation with 82 annotated images as validation data (supervised method) and then tested through raw images (unannotated) as test data. Meanwhile, the K-means models were trained which led to machine-generated color classes of coffee fruit for semi-supervised image annotation. Consequently, the developed model efficiently analyzed the test data with an mAP@.5 (mean average precision) of 0.89. Strikingly, our innovative semi-supervised method with an mAP@.5 of 0.77 for multi-class mode surpassed the supervised method which had mAP@.5 of only 0.60, leading to faster and more accurate annotation. While testing the yield estimation of the model in two plots, an average error of 3.78% was recorded between the predicted (pre-harvest) and ground truth (harvest) data for binary class mode. The average error for multi-class data was recorded as 3.87% for green, 3.445% for green-yellow, 5.09% for cherry-raisin, and 2.51% for dry fruits. This AI-based technology when integrated with other tools such as UAV would efficiently remotely monitor coffee field for informed decision about irrigation, fertilizer application and other measures of timely field management, hence advancing precision agriculture. Moreover, this machine learning intelligent model can be tailored for various other fruit farming.