Standard Classification Algorithms Research Articles

NCLWO: Newton’s cooling law-based weighted oversampling algorithm for imbalanced datasets with feature noise

Imbalanced datasets pose challenges to standard classification algorithms. Although oversampling techniques can balance the number of samples across different classes, the difficulties of imbalanced classification is not solely imbalanced data itself but other factors, such as small disjuncts and overlapping regions, especially in the presence of noise. Traditional oversampling techniques are not effectively address these intricacies. To this end, we propose a novel oversampling method called Newton’s Cooling Law-Based Weighted Oversampling (NCLWO). The proposed method initially calculates the weight of the minority class based on density and closeness factors to identify hard-to-learn samples, assigning them higher heat. Subsequently, Newton’s Cooling Law is applied to each minority class sample by using it as the center and expanding the sampling region outward, gradually decreasing the heat until reaching a balanced state. Finally, majority class samples within the sampling region are translated to eliminate overlapping areas, and a weighted oversampling approach is employed to synthesize informative minority class samples. The experimental study, carried out on a set of benchmark datasets, confirm that the proposed method not only outperforms state-of-the-art oversampling approaches but also shows greater robustness in the presence of feature noise.

CARBO: Clustering and rotation based oversampling for class imbalance learning

Class imbalance of a data set is a crucial problem in machine learning where one class significantly outnumbers others. In such a data set, classification is a troublesome task for the standard classification algorithms, leading to bias towards the majority class. Different methods have been developed so far, such as oversampling, undersampling, and cost-sensitive learning, to deal with class imbalance circumstances. Among these techniques, oversampling technique does not suffer from the information loss and critical cost selection challenges. However, appropriate synthetic sample generation can be challenging and vulnerable to privacy leakage. This research proposed an oversampling technique, called CARBO, using threshold-based geometric rotation and majority class influenced clustering. Unlike the existing resampling approaches to class imbalance problem, we contribute to consider the data privacy and optimal sample generation together for effective oversampling. The performance of CARBO is evaluated using 44 benchmark imbalanced data set. The empirical analysis elucidates that CARBO can make boosting-based C4.5 ensemble classifiers perform higher for 73% of the data set than six state-of-the-art approaches. In addition, the theoretical compatibility analysis of CARBO demonstrates its robustness.

Residual vector and solution mode analysis using semi-supervised machine learning for mesh modification and CFD stability improvement

Novel methods are studied to improve the performance of our previous mesh optimization approach for the stability improvement of unstructured-mesh finite-volume simulations. The residual vector as well as solution modes from Principal Component Analysis of solution vectors is analyzed for this purpose. After sufficient growth, instabilities appear as anomalies in the dominant numerical modes. Using standard classification algorithms, such outliers can be detected readily and much more efficiently compared to the eigenanalysis of the Jacobian matrix, which was required by the forebears of the current study. Further, it is essential to identify the correct local regions on the mesh for possible modification of vertex location and to remove the noise from the non-related cells. Hence, a synthetic vector is constructed from the working vector containing instabilities to simulate the behavior of the right eigenvectors in the solution. The results show the feasibility of residual vector analysis and principal component analysis of solution vectors for the stability improvement of finite-volume simulations. The new approach results in complete automation of the mesh optimization application.

Ensemble Predictors: Possibilistic Combination of Conformal Predictors for Multivariate Time Series Classification.

In this article we propose a conceptual framework to study ensembles of conformal predictors (CP), that we call Ensemble Predictors (EP). Our approach is inspired by the application of imprecise probabilities in information fusion. Based on the proposed framework, we study, for the first time in the literature, the theoretical properties of CP ensembles in a general setting, by focusing on simple and commonly used possibilistic combination rules. We also illustrate the applicability of the proposed methods in the setting of multivariate time-series classification, showing that these methods provide better performance (in terms of both robustness, conservativeness, accuracy and running time) than both standard classification algorithms and other combination rules proposed in the literature, on a large set of benchmarks from the UCR time series archive.

Which standard classification algorithm has more stable performance for imbalanced network traffic data?

Which standard classification algorithm has more stable performance for imbalanced network traffic data?

Accurate, automated taxonomic assignment of genebank accessions: a new method demonstrated using high-throughput marker data from 10,000 Capsicum spp. accessions

Key messageWe demonstrate how an algorithm that uses cheap genetic marker data can ensure the taxonomic assignments of genebank samples are complete, intuitive, and consistent—which enhances their value.To maximise the benefit of genebank resources, accurate and complete taxonomic assignments are imperative. The rise of genebank genomics allows genetic methods to be used to ensure this, but these need to be largely automated since the number of samples dealt with is too great for efficient manual recategorisation, however no clearly optimal method has yet arisen. A recent landmark genebank genomic study sequenced over 10,000 genebank accessions of peppers (Capsicum spp.), a species of great commercial, cultural, and scientific importance, which suffers from much taxonomic ambiguity. Similar datasets will, in coming decades, be produced for hundreds of plant taxa, affording a perfect opportunity to develop automated taxonomic correction methods in advance of the incipient genebank genomics explosion, alongside providing insights into pepper taxonomy in general. We present a marker-based taxonomic assignment approach that combines ideas from several standard classification algorithms, resulting in a highly flexible and customisable classifier suitable to impose intuitive assignments, even in highly reticulated species groups with complex population structures and evolutionary histories. Our classifier performs favourably compared with key alternative methods. Possible sensible alterations to pepper taxonomy based on the results are proposed for discussion by the relevant communities.

A class-specific feature selection and classification approach using neighborhood rough set and K-nearest neighbor theories

Rough set theories are utilized in class-specific feature selection to improve the classification performance of continuous data while handling data uncertainty. However, most of those approaches are converting continuous data into discrete or fuzzy data before applying rough set theories. These data conversions can reduce or change the meaning of data, as well as introduce unnecessary complexity to the feature domain. Therefore, in this study, we use neighborhood rough sets in class-specific feature selection to improve the classification performance without data conversions. As the standard classification algorithms are capable of handling a single feature set, we propose a novel classification algorithm based on the K-Nearest Neighbour algorithm to use class-specific feature subsets. Experimental evaluations prove that the proposed approach outperforms most of the baselines, and the selected feature sets are more effective than using the full feature sets in classification. The approach highly reduces the number of selected features and hence, can be used for effective data analysis of continuous data with high performance.

Application of KM-SMOTE for rockburst intelligent prediction

Class-imbalanced is a common phenomenon in rockburst data, and the prediction of rockburst intensity through intelligent methods requires a balanced dataset. This fact presents challenges for standard classification algorithms that are designed for class distributions that are well-balanced. This paper develops the modified synthetic minority oversampling technique by K-means cluster (KM-SMOTE) to reduce the imbalance phenomenon in the rockburst dataset. First, the study collects 226 rockburst cases worldwide as the original supporting dataset and selects four indexes to predict the rockburst intensity, namely, the maximum tangential stress of the surrounding rock σθ, the uniaxial compressive strength of rock σc, the tensile strength of rock σt, and the elastic energy index Wet. Second, the KM-SMOTE uses a K-means cluster to cluster the minority-class samples and then performs SMOTE oversampling on each cluster to obtain 388 data. To establish a nonlinear correlation between rockburst intensity and its predictors, six machine-learning classifiers are used. The dataset is randomly divided into training and test sets, with 80% of the data used for training. In the data training and testing phases, the original dataset, SMOTE-processed dataset, and KM-SMOTE-processed dataset were put into the machine learning models for predicting rockburst intensity, where KM-SMOTE was 3.3% and 10.5% more accurate than the SMOTE-processed dataset in predicting rockburst intensity, respectively. In the Jiangbian Hydropower Station engineering application, the KM-SMOTE algorithm can achieve a maximum improvement of 25% in accuracy compared with the data processed by SMOTE. Overall, the proposed modified oversampling algorithm effectively overcomes class-imbalanced in the rockburst dataset and significantly contributes to the intelligent prediction of rockburst by machine learning in engineering.

Heart Failure Detection Using Instance Quantum Circuit Approach and Traditional Predictive Analysis

The earlier prediction of heart diseases and appropriate treatment are important for preventing cardiac failure complications and reducing the mortality rate. The traditional prediction and classification approaches have resulted in a minimum rate of prediction accuracy and hence to overcome the pitfalls in existing systems, the present research is aimed to perform the prediction of heart diseases with quantum learning. When quantum learning is employed in ML (Machine Learning) and DL (Deep Learning) algorithms, complex data can be performed efficiently with less time and a higher accuracy rate. Moreover, the proposed ML and DL algorithms possess the ability to adapt to predictions with alterations in the dataset integrated with quantum computing that provides robustness in the earlier detection of chronic diseases. The Cleveland heart disease dataset is being pre-processed for the checking of missing values to avoid incorrect predictions and also for improvising the rate of accuracy. Further, SVM (Support Vector Machine), DT (Decision Tree) and RF (Random Forest) are used to perform classification. Finally, disease prediction is performed with the proposed instance-based quantum ML and DL method in which the number of qubits is computed with respect to features and optimized with instance-based learning. Additionally, a comparative assessment is provided for quantifying the differences between the standard classification algorithms with quantum-based learning in order to determine the significance of quantum-based detection in heart failure. From the results, the accuracy of the proposed system using instance-based quantum DL and instance-based quantum ML is found to be 98% and 83.6% respectively.

End-to-End: A Simple Template for the Long-Tailed-Recognition of Transmission Line Clamps via a Vision-Language Model

Raw image classification datasets generally maintain a long-tailed distribution in the real world. Standard classification algorithms face a substantial issue because many labels only relate to a few categories. The model learning processes will tend toward the dominant labels under the influence of their loss functions. Existing systems typically use two stages to improve performance: pretraining on initial imbalanced datasets and fine-tuning on balanced datasets via re-sampling or logit adjustment. These have achieved promising results. However, their limited self-supervised information makes it challenging to transfer such systems to other vision tasks, such as detection and segmentation. Using large-scale contrastive visual-language pretraining, the Open AI team discovered a novel visual recognition method. We provide a simple one-stage model called the text-to-image network (TIN) for long-tailed recognition (LTR) based on the similarities between textual and visual features. The TIN has the following advantages over existing techniques: (1) Our model incorporates textual and visual semantic information. (2) This end-to-end strategy achieves good results with fewer image samples and no secondary training. (3) By using seesaw loss, we further reduce the loss gap between the head category and the tail category. These adjustments encourage large relative magnitudes between the logarithms of rare and dominant labels. TIN conducted extensive comparative experiments with a large number of advanced models on ImageNet-LT, the largest long-tailed public dataset, and achieved the state-of-the-art for a single-stage model with 72.8% at Top-1 accuracy.

Multiple Imputation Ensembles for Time Series (MIE-TS)

Time series classification has become an interesting field of research, thanks to the extensive studies conducted in the past two decades. Time series may have missing data, which may affect both the representation and also modeling of time series. Thus, recovering missing data using appropriate time series-based imputation methods is an essential step. Multiple imputation is a data recovery method where it produced multiple imputed data. The method proves its usefulness in terms of reflecting the uncertainty inherit in missing data; however, it is under-researched in time series problems. In this article, we propose two multiple imputation approaches for time series. The first is a multiple imputation method based on interpolation. The second is a multiple imputation and ensemble method. First, we simulate missing consecutive sub-sequences under a Missing Completely at Random mechanism; then, we use single/multiple imputation methods. The imputed data are used to build bagging and stacking ensembles. We build ensembles using standard classification algorithms as well as time series classifiers. The standard classifiers involve Random Forest, Support Vector Machines, K-Nearest Neighbour, C4.5, and PART while TSCHIEF, Proximity Forest, Time Series Forest, RISE, and BOSS are chosen as time series classifiers. Our findings show that the combination of multiple imputation and ensemble improves the performance of the majority of classifiers tested in this study, often above the performance obtained from the complete data, even under increasing missing data scenarios. This may be because the diversity injected by multiple imputation has a very favourable and stabilising effect on the classifier performance, which is a very important finding.

Inverse free reduced universum twin support vector machine for imbalanced data classification

Imbalanced datasets are prominent in real-world problems. In such problems, the data samples in one class are significantly higher than in the other classes, even though the other classes might be more important. The standard classification algorithms may classify all the data into the majority class, and this is a significant drawback of most standard learning algorithms, so imbalanced datasets need to be handled carefully. One of the traditional algorithms, twin support vector machines (TSVM), performed well on balanced data classification but poorly on imbalanced datasets classification. In order to improve the TSVM algorithm’s classification ability for imbalanced datasets, recently, driven by the universum twin support vector machine (UTSVM), a reduced universum twin support vector machine for class imbalance learning (RUTSVM) was proposed. The dual problem and finding classifiers involve matrix inverse computation, which is one of RUTSVM’s key drawbacks. In this paper, we improve the RUTSVM and propose an improved reduced universum twin support vector machine for class imbalance learning (IRUTSVM). We offer alternative Lagrangian functions to tackle the primal problems of RUTSVM in the suggested IRUTSVM approach by inserting one of the terms in the objective function into the constraints. As a result, we obtain new dual formulation for each optimization problem so that we need not compute inverse matrices neither in the training process nor in finding the classifiers. Moreover, the smaller size of the rectangular kernel matrices is used to reduce the computational time. Extensive testing is carried out on a variety of synthetic and real-world imbalanced datasets, and the findings show that the IRUTSVM algorithm outperforms the TSVM, UTSVM, and RUTSVM algorithms in terms of generalization performance.

Algorithmic TCAMs: Implementing Packet Classification Algorithms in Hardware

The adoption of cloud computing and the deployment of new services require more and more computing and networking resources in large data centers. In particular, there is a need for faster, rich featured switches. Today, switching integrated circuits can handle more than 50 Tb/s and are expected to reach 100 Tb/s next year. This trend is likely to continue, approaching petabit capacities in a few years. At the same time, switches need to support more advanced packet classification features. This poses many challenges to the designers of switching integrated circuits, creating a need for innovations to move forward. One of those challenges is to support programmable packet classification with large rulesets at wire speed using limited silicon footprint. Unfortunately, traditional solutions like ternary content addressable memories (TCAMs) have high cost in area and power, and therefore are not a viable option for large on-chip rulesets, so new alternatives are needed. In the last two decades, researchers have extensively studied the problem of packet classification, proposing many algorithms based on standard memories, mostly targeting software implementations. These packet classification algorithms could be a solution to enable large on-chip classification rulesets using standard memories. However, porting the standard packet classification algorithms to a hardware implementation is not a trivial task, due to the different requirements and the different kinds of memory and computing resources available on application-specific integrated circuit (ASIC) switching chips. This article describes a packet classification design tailored for high-speed switching integrated circuits, which is currently used in the NVIDIA® Mellanox® Spectrum® switching ASIC product line, illustrating how to solve the mismatch between the usual software-based classification algorithms and the specific requirements of a hardware implementation.

Point cloud-based pig body size measurement featured by standard and non-standard postures

Automatic body size measurement of livestock using 3D vision technology is one of the current research focuses. Point clouds obtained from freely walking livestock involve a variety of postures. However, measuring livestock body sizes requires their upright and straight standard postures, and non-standard measurement postures such as bending their body or lowering their head lead to inaccuracy in body size estimation. In this paper, we analyze 207 groups of point cloud data collected from 25 landrace pigs, and propose a standard posture classification algorithm. By using the algorithm based on the median skeleton extraction, 22 key points of the pig's skeleton were obtained and all joint point vector eigenvalue sets were calculated. Then Bagging-SVM Posture Classifier was constructed, and by three experts’ votes, comparisons were made between the automatic classification results and the manual classification results based on standard and non-standard postures of pigs. According to the automatic calculation of four parameters including body length, body height, body width and abdominal girth in standard and non-standard postures, the experimental data showed that the body size measurement results in standard postures presented high stability and consistency, while the results in non-standard postures fluctuated considerably, which significantly affected the accuracy and stability. The experiment showed that the ratio of standard postures to non-standard postures was 1:3. Finally, based on joint point vectors eigenvalues, the regression models were applied to adjust body length, body height, body width and abdominal girth in non-standard postures. The linear regression and nonlinear regression methods such as BP regression and SVR support vector regression were used for comparison. The experimental results indicated that various regression methods can significantly improve the accuracy of automatic measurement by correcting the pig's body size measurement results in non-standard postures. Therefore, it is essential to classify and adjust the livestock point cloud postures to improve the accuracy of three-dimensional measurement.

Predicting severely imbalanced data disk drive failures with machine learning models

Datasets related to hard drive failure, particularly BackBlaze Hard Drive Data, have been widely studied in the literature using many statistical, machine learning, and deep learning techniques. These datasets are severely imbalanced due to the presence of a small number of failed drives compared to huge amounts of healthy drives in the operational data centers. It is challenging to mitigate the adverse consequence of the class imbalance due to the presence of bias towards the majority class during learning. SMART (self monitoring analysis and reporting technology) attributes of the disk drives were utilized in the past to design standard classification or regression algorithms. Although few machine learning (ML) models, for instance, tree based methods and ensemble learning algorithms, addressed the failure prediction, the effects of class imbalance were rarely properly considered under the ML framework. This study, based on a review of the state-of-the-art in the area, evaluates current methodologies to identify areas that were either overlooked or lacking, proposes methods for remediating these issues, and performs some baseline experiments to demonstrate the proposed methodologies including data sampling techniques and cost-sensitive learning.

Using Credal C4.5 for Calibrated Label Ranking in Multi-Label Classification

Multi-Label Classification (MLC) assumes that each instance belongs to a set of labels, unlike traditional classification, where each instance corresponds to a unique value of a class variable. Calibrated Label Ranking (CLR) is an MLC algorithm that determines a ranking of labels for a given instance by considering a binary classifier for each pair of labels. In this way, it exploits pairwise label correlations. Furthermore, CLR alleviates the class-imbalance problem that usually arises in MLC because, in this domain, very few instances often belong to a label. In order to build the binary classifiers in CLR, it is required to employ a standard classification algorithm. The Decision Tree method C4.5 has been widely used in this field. In this research, we show that a version of C4.5 based on imprecise probabilities recently proposed, known as Credal C4.5, is more appropriate than C4.5 to handle the binary classification tasks in CLR. Experimental results reveal that Credal C4.5 outperforms C4.5 when both methods are used in CLR and that the difference is more statistically significant as the label noise level is higher.

A Novel Walking Activity Recognition Model for Rotation Time Series Collected by a Wearable Sensor in a Free-Living Environment.

Solutions to assess walking deficiencies are widespread and largely used in healthcare. Wearable sensors are particularly appealing, as they offer the possibility to monitor gait in everyday life, outside a facility in which the context of evaluation biases the measure. While some wearable sensors are powerful enough to integrate complex walking activity recognition models, non-invasive lightweight sensors do not always have the computing or memory capacity to run them. In this paper, we propose a walking activity recognition model that offers a viable solution to this problem for any wearable sensors that measure rotational motion of body parts. Specifically, the model was trained and tuned using data collected by a motion sensor in the form of a unit quaternion time series recording the hip rotation over time. This time series was then transformed into a real-valued time series of geodesic distances between consecutive quaternions. Moving average and moving standard deviation versions of this time series were fed to standard machine learning classification algorithms. To compare the different models, we used metrics to assess classification performance (precision and accuracy) while maintaining the detection prevalence at the level of the prevalence of walking activities in the data, as well as metrics to assess change point detection capability and computation time. Our results suggest that the walking activity recognition model with a decision tree classifier yields the best compromise in terms of precision and computation time. The sensor that was used had purposely low computing and memory capacity so that reported performances can be thought of as the lower bounds of what can be achieved. Walking activity recognition is performed online, i.e., on-the-fly, which further extends the range of applicability of our model to sensors with very low memory capacity.

N-GRAMS ASSISTED YOUTUBE SPAM COMMENTDETECTION

This paper proposes a novel technique for detecting intrusive comments or spam on the video sharing website - YouTube. We describe spam comments as those which have persuasive intent or those who deem to be contextually irrelevant for a given video. The prospects of monetization through advertising on popular social media channels over the years have attracted an increasingly large number of users. This has in turn led to the growth of the malicious users who have begun to build up automated bots, which are proficient of large scale orchestrated distribution of spam messages across multiple channels simultaneously. These intrusive comments damage the fame of a channel and also the experience of regular users. YouTube themselves have embarked upon this issue with some finite methods which blocks unsolicited comments that consists of links. Those methods have proven to be exceptionally unproductive as spammers have found strategies to bypass such heuristics. Standard machine learning classification algorithms are operative but there is always a possibility for better accuracy with novel methods. In this work, we aim to identify such comments by implementing conventional machine learning algorithms such as Random Forest, Support Vector Machine, and Naive Bayes along with certain custom heuristics such as Count Vectorizer which have proven to be very effective in detecting and subsequently combating spam commentary.

FEM simulation based adaptive sensors array error compensation method for impact monitoring on stiffened composite structures

To eliminate the sensor array based impact localization bias on stiffened composite structures, a finite element method (FEM) simulation based adaptive sensors array errors compensation method for impact monitoring is introduced. Firstly, detailed analysis and comparison discussion about Lamb wave propagation errors between the simple and stiffened plate are provided using FEM simulation, and the gain-phase errors obtained are structured as a array error matrix. Secondly, array error matrix is substituted into the steering vector of standard two-dimensional Multiple Signal Classification (2D-MUSIC) algorithm to compensate the stiffener effect. And then initial estimates of impact could be estimated by the compensated 2D-MUSIC algorithm. Finally substituting these initial estimates, the cost function is minimized by adaptive iterative to calculate the reasonable location of the impact source. The experiments on a carbon fiber composite stiffened plate demonstrate the validity and effectiveness of the proposed method.

MCBC-SMOTE: A Majority Clustering Model for Classification of營mbalanced Data

Datasets with the imbalanced class distribution are difficult to handle with the standard classification algorithms. In supervised learning, dealing with the problem of class imbalance is still considered to be a challenging research problem. Various machine learning techniques are designed to operate on balanced datasets; therefore, the state of the art, different under-sampling, over-sampling and hybrid strategies have been proposed to deal with the problem of imbalanced datasets, but highly skewed datasets still pose the problem of generalization and noise generation during resampling. To over-come these problems, this paper proposes a majority clustering model for classification of imbalanced datasets known as MCBC-SMOTE (Majority Clustering for balanced Classification-SMOTE). The model provides a method to convert the problem of binary classification into a multi-class problem. In the proposed algorithm, the number of clusters for the majority class is calculated using the elbow method and the minority class is over-sampled as an average of clustered majority classes to generate a symmetrical class distribution. The proposed technique is cost-effective, reduces the problem of noise generation and successfully disables the imbalances present in between and within classes. The results of the evaluations on diverse real datasets proved to provide better classification results as compared to state of the art existing methodologies based on several performance metrics.