Instances In Classes Research Articles

Cost-Sensitive Boosting Algorithm for Classifying Underdeveloped Regions in Indonesia

Imbalanced classes are indicated by having more instances of some classes than others. The cost-sensitive boosting algorithm is a modification of the AdaBoost algorithm, which aims to solve the problem of imbalanced classes. In this study, we evaluate the cost-sensitive Boosting algorithm AdaC2 using Indonesia's underdeveloped region's data. This study confirms that the cost-sensitive boosting algorithm (AdaC2) performs better in classifying the instances in the minority classes than standard classifiers algorithms.

VGG16 Deep Learning Architecture Using Imbalance Data Methods For The Detection Of Apple Leaf Diseases

Data in the real world, there are many conditions (situations) where the number of instances in one class is much less than the number of instances in other classes. This situation is a problem in unbalanced datasets (imbalance class). As a result, performance in classification will decrease in some data systems. In this study, it was identified that the apple leaf disease performance dataset used had a large enough data imbalance problem where the comparison between instances was 1:5, so an oversampling method was needed to solve the data imbalance problem. Methods that can be used include the Synthetic Minority Over Sampling Technique (SMOTE). In order to validate the effectiveness of the proposed model, two experimental scenarios were carried out: first, the VGG16 algorithm was directly applied to modeling without considering class imbalance by reducing the number of layers and kernels in each layer to achieve optimal results, second, over-sampling SMOTE to increase the number of balanced datasets. The results showed that using the confusion matrix the accuracy results for each method were obtained where VGG 16 scored 85.16%, VGG 16 with SMOTE scored 92.94%. The conclusion of this study is that SMOTE helps improve the accuracy of leaf disease detection in apples.

Performance analysis and comparison of various machine learning algorithms for early stroke prediction

AbstractStroke is the leading cause of permanent disability in adults, and it can cause permanent brain damage. According to the World Health Organization, 795 000 Americans experience a new or recurrent stroke each year. Early detection of medical disorders, for example, strokes, can minimize the disabling effects. Thus, in this paper, we consider various risk factors that contribute to the occurrence of stoke and machine learning algorithms, for example, the decision tree, random forest, and naive Bayes algorithms, on patient characteristics survey data to achieve high prediction accuracy. We also consider the semisupervised self‐training technique to predict the risk of stroke. We then consider the near‐miss undersampling technique, which can select only instances in larger classes with the smaller class instances. Experimental results demonstrate that the proposed method obtains an accuracy of approximately 98.83% at low cost, which is significantly higher and more reliable compared with the compared techniques.

The Challenge of Data Annotation in Deep Learning-A Case Study on Whole Plant Corn Silage.

Recent advances in computer vision are primarily driven by the usage of deep learning, which is known to require large amounts of data, and creating datasets for this purpose is not a trivial task. Larger benchmark datasets often have detailed processes with multiple stages and users with different roles during annotation. However, this can be difficult to implement in smaller projects where resources can be limited. Therefore, in this work we present our processes for creating an image dataset for kernel fragmentation and stover overlengths in Whole Plant Corn Silage. This includes the guidelines for annotating object instances in respective classes and statistics of gathered annotations. Given the challenging image conditions, where objects are present in large amounts of occlusion and clutter, the datasets appear appropriate for training models. However, we experience annotator inconsistency, which can hamper evaluation. Based on this we argue the importance of having an evaluation form independent of the manual annotation where we evaluate our models with physically based sieving metrics. Additionally, instead of the traditional time-consuming manual annotation approach, we evaluate Semi-Supervised Learning as an alternative, showing competitive results while requiring fewer annotations. Specifically, given a relatively large supervised set of around 1400 images we can improve the Average Precision by a number of percentage points. Additionally, we show a significantly large improvement when using an extremely small set of just over 100 images, with over 3× in Average Precision and up to 20 percentage points when estimating the quality.

A weighted hybrid ensemble method for classifying imbalanced data

In real datasets, most are unbalanced. Data imbalance can be defined as the number of instances in some classes greatly exceeds the number of instances in other classes. Whether in the field of data mining or machine learning, data imbalance can have adverse effects. At present, the methods to solve the problem of data imbalance can be divided into data-level methods, algorithm-level methods and hybrid methods. In this paper, we propose a weighted hybrid ensemble method for classifying imbalanced data in binary classification tasks, called WHMBoost. In the framework of the boosting algorithm, the presented method combines two data sampling methods and two base classifiers, and each sampling method and each base classifier is assigned corresponding weights, which makes them have better complementary advantages. The performance of WHMBoost has been evaluated on 40 benchmark imbalanced datasets with state of the art ensemble methods like AdaBoost, RUSBoost, SMOTEBoost using AUC, F-Measure and Geometric Mean as the performance evaluation criteria. Experimental results show significant improvement over the other methods and it can be concluded that WHMBoost is a promising and effective algorithm to deal with imbalance datasets.

NearCount: Selecting critical instances based on the cited counts of nearest neighbors

Traditional instance selection algorithms are not good at addressing imbalanced problems. Moreover, most of them are sensitive to noise instances and suffer from complex selection rules. To solve these problems, in this paper, we propose a concise learning framework named NearCount to determine the importance of the instance without editing noise. In NearCount, the importance of an instance corresponds to the cited counts. The count is determined by the number of times that one instance is selected as a nearest neighbor of instances in different classes. For the instances with nonzero cited counts, the importance of the instance is inversely proportional to the cited count. To handle classification problems with different data distributions, two detailed NearCount-based algorithms – NearCount-IM and NearCount-IS – are introduced. For imbalanced problems, NearCount-IM selects the important majority instances with an equal number of minority instances, thus balancing the data distribution. For balanced scenarios, NearCount-IS selects the instances whose cited counts are greater than zero and equal or less than the number of nearest neighbors as critical instances in every class. The proposed NearCount-IM and NearCount-IS algorithms are evaluated by comparing them with classical instance selection algorithms on the benchmark data sets. Experiments validate the effectiveness of the proposed algorithms.

MFSC: Multi-voting based feature selection for classification of Golgi proteins by adopting the general form of Chou's PseAAC components

Automatic identification of protein subcellular localization has gained much popularity in the last few decades. Subcellular localizations are useful in diagnosis of different diseases as well as in the process of drug development. Golgi is a vital type of protein, which provides means of transportation to several other proteins destined for lysosome, plasma membrane and secretion etc. Cis-Golgi and trans-Golgi are two ends of Golgi protein meant for reception and transmission of various substances. Dysfunction in Golgi proteins may lead to different types of diseases especially the inheritable and neurodegenerative problems.Due to the significance of Golgi proteins, it is indispensable to correctly identify the Golgi proteins. In this paper, a novel and high throughput computational model is proposed which can identify the subGolgi proteins precisely. Discrete and evolutionary feature extraction schemes are applied so that all the salient, noiseless, and relevant information from protein sequences could be captured. Unfortunately, the benchmark dataset publicly available is quite imbalance, where trans-Golgi sequences constitute 72% of the whole dataset that reflects biasness, redundancy, and lack of hypothesis generalization. In order to cover the limitations of imbalance data, Synthetic Minority over Sampling Technique is utilized to balance the number of instances in different classes of the dataset. In addition, a condense feature space is formed by fusing the high rank features of eleven different feature selection techniques. The high rank features are selected through majority voting algorithm; consequently, the feature space is reduced 85%. The experiential results demonstrate that kNN classifier obtained promising results in combination with hybrid feature space. It has yielded an accuracy of 98% in jackknife cross-validation, 94% in independent data and 96% in 10-fold cross-validation test. It is ascertained that the proposed model is reliable, consistent and serves as a valuable tool for the research community.

Optimization Model of K-Means Clustering Using Artificial Neural Networks to Handle Class Imbalance Problem

Class imbalance is a situation where instances in one class much higher than instances in other classes. In clustering, this problem not only affects the accuracy of a prediction but also introduces bias in decision-making process. In this case, a machine learning technique will yield a good prediction accuracy from training data class with a large number of instances, but give a poor accuracy in classes with the small number of instances. In this research, we propose an approach for optimizing K-Means clustering in handling class imbalance problem. The approach uses the perceptron feed-forward neural network to determine coordinates of the centroid of a cluster in K-Means clustering processes. Data used in this research are datasets from the UCI Machine Learning Repository. From the experimental results obtained, the proposed approach could optimize the result of K-Means clustering in terms of minimizing class imbalance.

Using Receiver Operating Characteristic Curves To Optimize Discovery-Based Software with Comprehensive Two-Dimensional Gas Chromatography with Time-of-Flight Mass Spectrometry.

We report a quantitative approach to optimize implementation of discovery-based software for comprehensive two-dimensional gas chromatography coupled with time-of-flight mass spectrometry (GC × GC-TOFMS). The software performs a tile-based Fisher ratio (F-ratio) analysis and facilitates a supervised nontargeted analysis based upon the experimental design to aid in the discovery of analytes with statistically different variances between sample classes. The quantitative approach for software optimization uses receiver operating characteristic (ROC) curves. The area under the curve (AUC) for each ROC curve serves as a quantitative metric to optimize two key algorithm parameters: the signal-to-noise ratio (S/N) threshold of the data prior to calculating F-ratios at each m/z mass channel and the number of these F-ratios per m/z used to calculate the average F-ratio of a tile. A total of 25 combinations of S/N threshold by number of m/z were studied. Fifty analytes were spiked into a diesel fuel at two concentration levels to produce two sample classes that should in principle produce 50 positive instances in the ROC curves. The "sweet spot" for F-ratio analysis was determined to be a S/N threshold of 10 coupled with a maximum of the 10 most chemically selective m/z (requiring a minimum of 3 m/z), corresponding to an ∼21% improvement in the discrimination of true positives relative to prior studies. This equates to an additional 9 true positives being discovered at a false positive probability of 0.2 and 5 additional true positives being found overall. Furthermore, optimization of these software parameters did not depend upon a priori determination of the statistically correct number of positive instances in the sample classes. The AUC metric appears to be suitable for the evaluation of all data analysis methods that utilize the proper experimental design.

CogBoost: Boosting for Fast Cost-Sensitive Graph Classification

Graph classification has drawn great interests in recent years due to the increasing number of applications involving objects with complex structure relationships. To date, all existing graph classification algorithms assume, explicitly or implicitly, that misclassifying instances in different classes incurs an equal amount of cost (or risk), which is often not the case in real-life applications (where misclassifying a certain class of samples, such as diseased patients, is subject to more expensive costs than others). Although cost-sensitive learning has been extensively studied, all methods are based on data with instance-feature representation. Graphs, however, do not have features available for learning and the feature space of graph data is likely infinite and needs to be carefully explored in order to favor classes with a higher cost. In this paper, we propose, CogBoost, a fast cost-sensitive graph classification algorithm, which aims to minimize the misclassification costs (instead of the errors) and achieve fast learning speed for large scale graph data sets. To minimize the misclassification costs, CogBoost iteratively selects the most discriminative subgraph by considering costs of different classes, and then solves a linear programming problem in each iteration by using Bayes decision rule based optimal loss function. In addition, a cutting plane algorithm is derived to speed up the solving of linear programs for fast learning on large scale data sets. Experiments and comparisons on real-world large graph data sets demonstrate the effectiveness and the efficiency of our algorithm.

Classification of imbalanced bioinformatics data by using boundary movement-based ELM.

To address the imbalanced classification problem emerging in Bioinformatics, a boundary movement-based extreme learning machine (ELM) algorithm called BM-ELM was proposed. BM-ELM tries to firstly explore the prior information about data distribution by condensing all training instances into the one-dimensional feature space corresponding to the original output in ELM, and then on the transformed space, to find the optimal moving distance of the classification hyperplane by estimating the probability density distributions of the instances in different classes. Experimental results on four real imbalanced bioinformatics classification data sets indicated that the proposed BM-ELM algorithm outperforms some traditional bias correction algorithms due to it can greatly improve the sensitivity of the classification results with small loss of specificity as possible. Also, BM-ELM algorithm has presented better performance than the widely used support vector machine (SVM) classifier. The algorithm can be widely popularized in various large-scale bioinformatics applications.

Performance Improvement in the Pattern Classification of Nominal Data Sets Applying Multiple Correspondence Analysis

Classification is a supervised learning problem used to discriminate data instances in different classes. The solution to this problem is obtained through algorithms (classifiers) that look for patterns of relationships between classes in known cases, using these relationships to classify unknown cases. The performance of the classifiers depends substantially of the data types. In order to give proper treatment to nominal data, this paper shows that the application of previous transformations can substantially improve the performance of classifiers, bringing significant benefits to the result of the whole process of Knowledge Discovery in Databases (KDD). This paper uses three different data sets with nominal data and two well-known classifiers: the Linear Discriminant Analysis (LDA), and the Naïve-Bayes (NB). For data transformation, the paper applies an approach called Geometric Data Analysis (GDA). The GDA techniques compared in this paper are the traditional Principal Component Analysis (PCA) and the underexplored Multiple Correspondence Analysis (MCA). The results confirm the capability of the GDA transformation to improve the classification accuracy and attest the superiority of the MCA in comparison with its precursor, the PCA, when applied to nominal data.

Active Learning without Knowing Individual Instance Labels: A Pairwise Label Homogeneity Query Approach

Traditional active learning methods require the labeler to provide a class label for each queried instance. The labelers are normally highly skilled domain experts to ensure the correctness of the provided labels, which in turn results in expensive labeling cost. To reduce labeling cost, an alternative solution is to allow nonexpert labelers to carry out the labeling task without explicitly telling the class label of each queried instance. In this paper, we propose a new active learning paradigm, in which a nonexpert labeler is only asked “whether a pair of instances belong to the same class”, namely, a pairwise label homogeneity. Under such circumstances, our active learning goal is twofold: (1) decide which pair of instances should be selected for query, and (2) how to make use of the pairwise homogeneity information to improve the active learner. To achieve the goal, we propose a “Pairwise Query on Max-flow Paths” strategy to query pairwise label homogeneity from a nonexpert labeler, whose query results are further used to dynamically update a Min-cut model (to differentiate instances in different classes). In addition, a “Confidence-based Data Selection” measure is used to evaluate data utility based on the Min-cut model's prediction results. The selected instances, with inferred class labels, are included into the labeled set to form a closed-loop active learning process. Experimental results and comparisons with state-of-the-art methods demonstrate that our new active learning paradigm can result in good performance with nonexpert labelers.

Online neural network model for non-stationary and imbalanced data stream classification

“Concept drift” and class imbalance are two challenges for supervised classifiers. “Concept drift” (or non-stationarity) is changes in the underlying function being learnt, and class imbalance is a vast difference between the numbers of instances in different classes of data. Class imbalance is an obstacle for the efficiency of most classifiers. Previous methods for classifying non-stationary and imbalanced data streams mainly focus on batch solutions, in which the classification model is trained using a chunk of data. Here, we propose an online Neural Network (NN) model. The NN model, is composed of two different parts for handling concept drift and class imbalance. Concept drift is handled with a forgetting function and class imbalance is handled with a specific error function which assigns different importance to error in separate classes. The proposed method is evaluated on 3 synthetic and 8 real world datasets. The results show statistically significant improvement to previous online NN methods.

Ensemble of online neural networks for non-stationary and imbalanced data streams

Concept drift (non-stationarity) and class imbalance are two important challenges for supervised classifiers. “Concept drift” (or non-stationarity) refers to changes in the underlying function being learnt, and class imbalance is a vast difference between the numbers of instances in different classes of data. Class imbalance is an obstacle for the efficiency of most classifiers. Research on classification of non-stationary and imbalanced data streams, mainly focuses on batch solutions, whereas online methods are more appropriate. Here, we propose an online ensemble of neural network (NN) classifiers. Ensemble models are the most frequent methods used for classifying non-stationary and imbalanced data streams. The main contribution is a two-layer approach for handling class imbalance and non-stationarity. In the first layer, cost-sensitive learning is embedded into the training phase of the NNs, and in the second layer a new method for weighting classifiers of the ensemble is proposed. The proposed method is evaluated on 3 synthetic and 8 real-world datasets. The results show statistically significant improvement compared to online ensemble methods with similar features.

Recursive least square perceptron model for non-stationary and imbalanced data stream classification

Classifying non-stationary and imbalanced data streams encompasses two important challenges, namely concept drift and class imbalance. “Concept drift” (or non-stationarity) is changes in the underlying function being learnt, and class imbalance is vast difference between the numbers of instances in different classes of data. Class imbalance is an obstacle for the efficiency of most classifiers and is usually observed in two-class datasets. Previous methods for classifying non-stationary and imbalanced data streams mainly focus on batch solutions, in which the classification model is trained using a chunk of data. Here, we propose an online perceptron model. The main contribution is a new error model inspired from the error model of recursive least square (RLS) filter. In the proposed error model, non-stationarity is handled with the forgetting factor of RLS error model and for handling class imbalance two different errors weighting strategies are proposed. These strategies are verified with convergence and tracking theories from adaptive filters theory. The proposed methods is evaluated on two synthetic and six real-world two-class datasets and compared with seven previous online perceptron models. The results show statistically significant improvement to previous methods.

Topological tori as abstract art

Topologists have known for almost three decades that there are exactly four regular homotopy classes for marked tori immersed in 3D Euclidean space. Instances in different classes cannot be smoothly transformed into one another without introducing tears, creases, or regions of infinite curvature into the torus surface. In this article, a quartet of simple, easy-to-remember representatives for these four classes is depicted to expose this classification to a broader audience. It is shown explicitly how the four models differ from one another, and examples are given of how a more complex immersion of a torus can be transformed into its corresponding representative. These mathematical visualization models are juxtaposed with artistic sculptures employing more classical torus shapes.

Online cost-sensitive neural network classifiers for non-stationary and imbalanced data streams

Classifying non-stationary and imbalanced data streams encompasses two important challenges, namely concept drift and class imbalance. Concept drift is changes in the underlying function being learnt, and class imbalance is vast difference between the numbers of instances in different classes of data. Class imbalance is an obstacle for the efficiency of most classifiers. Previous methods for classifying non-stationary and imbalanced data streams mainly focus on batch solutions, in which the classification model is trained using a chunk of data. Here, we propose two online classifiers. The classifiers are one-layer NNs. In the proposed classifiers, class imbalance is handled with two separate cost-sensitive strategies. The first one incorporates a fixed and the second one an adaptive misclassification cost matrix. The proposed classifiers are evaluated on 3 synthetic and 8 real-world datasets. The results show statistically significant improvements in imbalanced data metrics.

Feature Extraction with Space Folding Model and its Application to Machine Learning

Feature extraction provides an essential element in most machine learning methods, including supervised learning with neural networks. Linearly inseparable data distributions are often non-linearly transformed in some way to make them more linearly separable in the feature space. In this paper, we propose a method of feature extraction with a space folding model. In the proposed method, each basis vector in the m-dimensional data space is divided in the positive and negative directions to optimize it with 2mm-dimensional vectors as variables. 2mvariable vectors are estimated to minimize the cross entropy of class labels and distances so that instances in the same classes are gathered closer together and those in other classes are separated farther apart. The proposed method, in which linear transformation is applied to each quadrant to collectively realize a nonlinear transformation, is expected to lead to improvements in accuracy of discrimination over conventional methods of feature extraction using single linear transformations. In this paper, we have confirmed the effectiveness of the proposed method of feature extraction with a space folding model on a UCI benchmark problem.

Refining the phase transition in combinatorial search

Many recent studies have identified phase transitions from under- to overconstrained instances in classes of constraint satisfaction search problems, and their relation to search cost. For example, cases that are difficult to solve with a variety of search methods are concentrated near these transitions. These studies show that a simple parameter describing the problem structure predicts the difficulty of solving the problem, on average. However, this prediction is associated with a large variance and depends on the somewhat arbitrary choice of the problem class. Thus these results are of limited direct use for individual instances. To help address this limitation, additional parameters, describing problem structure as well as heuristic effectiveness, are introduced. These are shown to reduce the variation in some cases. This also provides further insight into the nature of intrinsically hard search problems.