Basic Classification Algorithm Research Articles

Evolutionary simultaneous under and oversampling of instances for dealing with class-imbalance datasets in multilabel problems

Multilabel classification has recently attracted great attention from the data mining research community. Multilabel classification is concerned with learning where each instance can be associated with multiple classes (or labels). Class-imbalance problems appear in any classification task when the class distribution of the instances is very different. In multilabel classification, this problem is ubiquitous, as a large percentage of labels suffer from a class-imbalanced distribution. The adaptation of single-label methods to deal with the class-imbalance problem in multilabel learning is problematic as many of their basic concepts are not easily transferred. In this paper, we propose the use of evolutionary computation to simultaneously oversample the minority class and undersample the majority class for multilabel problems. Letting the algorithm autonomously select the instances to undersample and oversample allows us to extend these two successful paradigms to the multilabel task. An extensive comparison setup of 35 datasets shows the advantages of using this approach to deal with class-imbalance datasets for multilabel problems compared with previously published methods as well as the basic classification algorithms with the original datasets.

Glove-Based Hand Gesture Recognition for Diver Communication.

We have developed a smart dive glove that recognizes 13 static hand gestures used in diving communication. The smart glove employs five dielectric elastomer sensors to capture finger motion and implements a machine learning classifier in the onboard electronics to recognize gestures. Five basic classification algorithms are trained and assessed: the decision tree, support vector machine (SVM), logistic regression, Gaussian naïve Bayes, and multilayer perceptron. These basic classifiers were selected as they perform well in multiclass classification problems, can be trained using supervised learning, and are model-based algorithms that can be implemented on a microprocessor. The training dataset was collected from 24 participants providing for a range of different hand sizes. After training, the algorithms were evaluated in a dry environment using data collected from ten new participants to test how well they cope with new information. Furthermore, an underwater experiment was conducted to assess any impact of the underwater environment on each algorithm's classification. The results show all classifiers performed well in a dry environment. The accuracies and F1-scores range between 0.95 and 0.98, where the logistic regressor and SVM have the highest scores for both the accuracy and F1-score (0.98). The underwater results showed that all algorithms work underwater; however, the performance drops when divers must focus on buoyancy control, breathing, and diver trim.

Creating a New Dataset for the Classification of Cyber Bullying

Regardless of young or old, people have quickly stepped into the world of internet with today's communication technologies such as phones, tablets, computers and smart devices. As the place of the Internet in people's lives increases, social media platforms are diversifying and users want to take part in these platforms. With the increase in the number of social media users, some negativities are encountered. The most important problem encountered in social media platforms is cyber bullying. Although cyber bullying seems to be a daily dialogue between social media users or between groups, the situation of encountering is increasing day by day with the diversity of shared information, content and agenda social media environments. With the development of technology, it is necessary to develop a platform that detects bullying with artificial intelligence technologies. One of the biggest difficulties in text classification problems that we encounter during the development of these platforms is the need to train the artificial intelligence algorithm to be used with labeled data. In this study, 21 different people, including journalists, athletes, scientists, doctors, politicians, comedians, social media phenomena, and artists who actively use social media, were selected in order to create the necessary dataset for training the models to be developed to detect cyber bullying situations. The public messages (mentions) of these 21 people sent via Twitter were compiled. After filtering the repetitive and meaningless messages sent by bot accounts out of 10500 tweets compiled, the number of messages in the dataset decreased to 7706. The labeling process, which is necessary for the dataset to be used for training and testing purposes in classification processes, was carried out by three independent people who were given preliminary information about cyberbullying (1=Includes Cyber bullying, 0=Does not include Cyber bullying). The majority of the tags, which were read and assigned by 3 different people, were accepted as the final class of the relevant message. Afterwards, the dataset was preprocessed in accordance with the principles of natural language processing and made suitable for classification algorithms. The findings obtained after the classification processes performed with the basic classification algorithms are shared. When the findings are examined, it is understood that the data set created has the competence to be used in the detection and prevention of cyber bullying. In this context, it is predicted that training specially developed and optimized artificial intelligence algorithms with the relevant dataset for the detection of cyberbullying will greatly increase the success rate.

METHOD OF BUILDING ENSEMBLES OF MODELS FOR DATA CLASSIFICATION BASED ON DECISION CORRELATIONS

The scientific work highlights the problem of increasing the accuracy of binary classification predictions using machine learning algorithms. Over the past few decades, systems that consist of many machine learning algorithms, also called ensemble models, have received increasing attention in the computational intelligence and machine learning community. This attention is well deserved, as ensemble systems have proven to be very effective and extremely versatile in a wide range of problem domains and real-world applications. One algorithm may not make a perfect prediction for a particular data set. Machine learning algorithms have their limitations, so creating a model with high accuracy is a difficult task. If you create and combine several models by combining and aggregating the results of each model, there is a chance to improve the overall accuracy, this problem is dealt with by ensembling. The basis of the information system of binary classification is the ensemble model. This model, in turn, contains a set of unique combinations of basic classifiers – a kind of algorithmic primitives. An ensemble model can be considered as some kind of meta-algorithm, which consists of unique sets of machine learning (ML) classification algorithms. The task of the ensemble model is to find such a combination of basic classification algorithms that would give the highest performance. The performance is evaluated according to the main ML metrics in classification tasks. Another aspect of scientific work is the creation of an aggregation mechanism for combining the results of basic classification algorithms. That is, each unique combination within the ensemble consists of a set of basic models (harbingers), the results of which must be aggregated. In this work, a non-hierarchical clustering method is used to aggregate (average) the predictions of the base models. A feature of this study is to find the correlation coefficients of the base models in each combination. With the help of the magnitude of correlations, the relationship between the prediction of the classifier (base model) and the true value is established, as a result of which space is opened for further research on improving the ensemble model (meta-algorithm)

Hybrid Graph Neural Network Model Design and Modeling Reasoning for Text Feature Extraction and Recognition

Natural language processing is an important branch of deep learning. In particular, the classification of short texts is one of the main tasks of computer linguistics, because it ensures the security of information. Therefore, this paper reviews the text classification methods for the first time, aiming at comparing the modern methods to solve the text classification problems, determine the trend in this direction, and select the best algorithm to be applied to research and business tasks. In this paper, with the help of the emerging algorithm in the field of deep learning, a web domain knowledge text classification method based on feature words is developed with the support vector machine (SVM) as the basic classification algorithm. Two datasets representing basic research disciplines and emerging research fields are selected to verify the proposed research framework. Experiments show that the performance of this method is improved by 3% compared with the benchmark method.

A New Model for Emotions Analysis in Social Network Text Using Ensemble Learning and Deep learning

Recently, emotion analysis has become widely used. Therefore, increasing the accuracy of existing methods has become a challenge for researchers. The proposed method in this paper is a hybrid model to improve the accuracy of emotion analysis; Which uses a combination of convolutional neural network and ensemble learning. In the proposed method, after receiving the dataset, the data is pre-processed and converted into process able samples. Then the new dataset is split into two categories of training and test. The proposed model is a structure for machine learning in the form of ensemble learning. It contains blocks consisting of a combination of convolutional networks and basic classification algorithms. In each convolutional network, the base classification algorithms replace the fully connected layer. Evaluate the proposed method, in IMDB, PL04 and SemEval dataset with accuracy, precision, recall and F1 criteria, shows that, on average, for all three datasets, the precision of polarity detection is 90%, the recall of polarity detection is 93%, the F1 of polarity detection is 91% and finally the accuracy of polarity detection is 92%.

A novel deep learning model based on convolutional neural networks for employee churn prediction

AbstractEmployees are one of the most important resources of a company. The churn of valuable employees significantly affects a company's performance. The design of systems that predict employee churn is critical importance for companies. At this point, machine learning algorithms offer important opportunities for the diagnosis of employee churn. Nowadays, traditional classification algorithms have been replaced by deep learning models. In this study, firstly, a convolutional neural network (CNN) model was applied on a numerical data set for employee churn prediction in retailing. Later, because the data loss is too much in data transformations, a new hybrid extended convolutional decision tree model (ECDT) was proposed by improving the CNN algorithm. Finally, a novel model (ECDT‐GRID) was developed by applying grid search optimization to improve the classification accuracy of ECDT. Numerical results showed that the developed ECDT‐GRID model outperformed the CNN and ECDT models and basic classification algorithms in terms of classification accuracy, and this model provided an efficient methodology for prediction of employee churn.

Bank Deposit Prediction Using Ensemble Learning

Bank deposit is one of the vital issues for any financial institution. It is very challenging to predict a customer if he/she can be a depositor by analyzing related information. Some recent reports demonstrate that economic depression and the continuous decline of the economy negatively impact business organizations and banking sectors. Due to such economic depression, banks cannot attract a customer's attention. Thus, marketing is preferred to be a handy tool for the banking sector to draw customers' attention for a term deposit. The purpose of this paper is to study the performance of ensemble learning algorithms which is a novel approach to predict whether a new customer will have a term deposit or not. A Portuguese retail bank data is used for our study, containing 45,211 phone contacts with 16 input attributes and one decision attribute. The data are preprocessed by using the Discretization technique. 40,690 samples are used for training the classifiers, and 4,521 samples are used for testing. In this work, the performance of the three mostly used classification algorithms named Support Vector Machine (SVM), Neural Network (NN), and Naive Bayes (NB) are analyzed. Then the ability of ensemble methods to improve the efficiency of basic classification algorithms is investigated and experimentally demonstrated. Experimental results exhibit that the performance metrics of Neural Network (Bagging) is higher than other ensemble methods. Its accuracy, sensitivity, and specificity are 96.62%, 97.14%, and 99.08%, respectively. Although all input attributes are considered in the classification method, in the end, a descriptive analysis has shown that some input attributes have more importance for this classification. Overall, it is shown that ensemble methods outperformed the traditional algorithms in this domain. We believe our contribution can be used as a depositor prediction system to provide additional support for bank deposit prediction.

The software complex for biochemical indicators monitoring taking into account ecological background of the region

This article is devoted review the process of using new methods of hypertonic disease monitoring. The authors suggest to use patient’s immunological and biochemical homeostasis for predicting and diagnosis this disease. It is proved that these data can be used for monitoring and controlling patients. The correlation between immuno-biochemical parameters and the ecological background patient’s place of residence are set. The problem of the design and construction of specialized complex laboratory control based on client-server architecture is considered. For data analysis supposed to be used statistical and intellectual processing methods. For example, in article describes the basic classification algorithm called “k nearest neighbors”. When the size of “training sample” is sufficient the accuracy in determining the class label reaches 99%. In conclusion emphasizes the importance of developing methods for early diagnosis of cardiovascular disease and using the modern methods for data analysis.

Predicting the revocation of a bank license using machine learning algorithms

This article presents the results of applying various machine learning methods to predict the revocation of credit organizations’ licenses in Russia. The goal of the research is to predict whether the bank’s license will be revoked soon. The feature space was analyzed, and additional features were calculated. Different basic classification algorithms, such as logistic regression, support vector machines classifier, decision tree, and bagging ensemble algorithm, were tested to solve the problem. Each algorithm was optimized to perform well with a highly unbalanced data. An enhanced bagging-based algorithm with weighted voting was developed to improve the classification quality. The results of this research can be used both by credit organizations themselves to monitor business conditions and assess risks, and by legal entities that cooperate with them for safe placement of their funds. A software tool in Python that allows solving problems of timely prediction of license revocation based on the developed algorithm was developed.

IMPT-FRAKEL: A Simple Multi-label Web-server that Only Uses Fingerprints to Identify which Metabolic Pathway Types Compounds can Participate In

Background: Metabolic pathway is one of the most basic biological pathways in living organisms. It consists of a series of chemical reactions and provides the necessary molecules and energies for organisms. To date, lots of metabolic pathways have been detected. However, there still exist hidden participants (compounds and enzymes) for some metabolic pathways due to the complexity and diversity of metabolic pathways. It is necessary to develop quick, reliable, and non-animal-involved prediction model to recognize metabolic pathways for any compound. Methods: In this study, a multi-label classifier, namely iMPT-FRAKEL, was developed for identifying which metabolic pathway types that compounds can participate in. Compounds and 12 metabolic pathway types were retrieved from KEGG. Each compound was represented by its fingerprints, which was the most widely used form for representing compounds and can be extracted from its SMILES format. A popular multi-label classification scheme, Random k-Labelsets (RAKEL) algorithm, was adopted to build the classifier. Classic machine learning algorithm, Support Vector Machine (SVM) with RBF kernel, was selected as the basic classification algorithm. Ten-fold cross-validation was used to evaluate the performance of the iMPT-FRAKEL. In addition, a web-server version of such classifier was set up, which can be assessed at http://cie.shmtu.edu.cn/impt/index. Results: iMPT-FRAKEL yielded the accuracy of 0.804, exact match of 0.745 and hamming loss of 0.039. Comparison results indicated that such classifier was superior to other models, including models with Binary Relevance (BR) or other classification algorithms. Conclusion: The proposed classifier employed limited prior knowledge of compounds but gives satisfying performance for recognizing metabolic pathways of compounds.

Hybrid classification algorithms based on instance filtering

Basic classification algorithms induce a single model from training data. The interpretation of a model is relatively easy, while basic algorithms have limitations in achieving high accuracy. An instance misclassified by a model may be correctly predicted by another. Hybrid classification is a concept that employs basic classification algorithms for model induction and for data preprocessing. Misclassification instances are usually considered to be noise, yet those still may carry useful information for identifying the class values of some other instances. This study proposes hybrid classification algorithms in which training instances are filtered to build three models for prediction. Each testing instance is classified by exactly one of them. The algorithms involved in the proposed hybrid classification algorithms are decision tree induction and naïve Bayesian classifier. The testing results on twenty data sets demonstrate that our hybrid classification algorithms can significantly outperform the basic ones as well as the hybrid algorithm proposed in a previous study. The hybrid classification algorithms based on instance filtering achieve relatively high accuracy and maintain the easy interpretation of learning results.

Investigating effects of force and pressure centre signals on stabilogram analysis

In this study, the centre of pressure (COP x ) and force change (F x ) in the anterior-posterior (AP) plane, the centre of pressure (COP y ) and force change (F y ) signals in medial-lateral (ML) plane were compared. In order to make comparisons in this context, the time-dependent norms of the COP x -COP y , and F x -F y signals (F B , COP B ) of 163 subjects were calculated separately. Subsequently, the mean standard deviation values (F STD , COP STD ) of these values generated by time-based were obtained. The scatter plots in the AP plane (F xSTD , COP xSTD ) and the ML plane (F ySTD , COP ySTD ) are presented in different categories, which depend on the standard deviation values, young-elderly of the measurement group, the firm-soft of the ground and open-close of the eyes. Finally, the separation successes of F xSTD - F ySTD and COP xSTD - COP ySTD were calculated by means of basic classification algorithms. According to the results obtained, it is observed that the force signals generally provide more successful results than COP signals. In the study, the highest accuracy was performed with force signals and the support vector machine algorithm separated the young-aged group with an accuracy of 81.67%.

IATC-NRAKEL: an efficient multi-label classifier for recognizing anatomical therapeutic chemical classes of drugs.

The anatomical therapeutic chemical (ATC) classification system plays an increasingly important role in drug repositioning and discovery. The correct identification of classes in each level of such system that a given drug may belong to is an essential problem. Several multi-label classifiers have been proposed in this regard. Although they provided satisfactory performance, the feature extraction procedures were still rough. More refined features may further improve the predicted quality. In this article, we provide a novel multi-label classifier, called iATC-NRAKEL, to predict drug ATC classes in the first level. To obtain more informative drug features, we employed the drug association information in STITCH and KEGG, which was organized by seven drug networks. The powerful network embedding algorithm, Mashup, was adopted to extract informative drug features. The obtained features were fed into the RAndom k-labELsets (RAKEL) algorithm with support vector machine as the basic classification algorithm to construct the classifier. The 10-fold cross-validation of the benchmark dataset with 3883 drugs showed that the accuracy and absolute true were 76.56 and 74.51%, respectively. The comparison results indicated that iATC-NRAKEL was much superior to all previous reported classifiers. Finally, the contribution of each network was analyzed. The codes of iATC-NRAKEL are available at https://github.com/zhou256/iATC-NRAKEL.

The Use of Artificial Neural Networks Optimized with Fire Fly Algorithm in Cancer Diagnosis

Today, the amount of biological data types obtained are increasing every day. Among these data types are micro arrays that play an important role in cancer diagnosis. The data analysis that are carried out through traditional approaches have proven unsuccessful in delivering efficient results on data types where data complexity is high and where sampling is low. For this reason, using a hybrid algorithm by merging the effective features of two distinct algorithms will yield effective results. In this study, a classification process was performed firstly by dimension reduction on micro array data that were obtained from the tissues from patients with a tumor in their central nervous system and then by using an artificial neural network algorithm that was optimized through Fire Fly Algorithm (FF), a hybrid approach. The data obtained were compared to K Nearest Neighbors (KNN), Support Vector Machine (SVM) and Artificial Neural Networks (ANN) classification algorithms, which are frequently used in the literature. Also, the results were compared to the findings that were obtained from artificial neural networks, which are reinforced by Genetic Algorithm (GA), another hybrid approach. Then the results were shared. The performance results obtained show that hybrid approaches present a highly precise and more efficient classification process but they show a slower performance than basic classification algorithms.

Ensemble Classification and Extended Feature Selection for Credit Card Fraud Detection

Due to the rise of technology, the possibility of fraud in different areas such as banking has been increased. Credit card fraud is a crucial problem in banking and its danger is over increasing. This paper proposes an advanced data mining method, considering both feature selection and decision cost for accuracy enhancement of credit card fraud detection. After selecting the best and most effective features, using an extended wrapper method, ensemble classification is performed. The extended feature selection approach includes a prior feature filtering and a wrapper approach using C4.5 decision tree. Ensemble classification, using cost sensitive decision trees is performed in a decision forest framework. A locally gathered fraud detection dataset is used to estimate the proposed method. The proposed method is assessed using accuracy, recall, and F-measure as evaluation metrics and compared with basic classification algorithms including ID3, J48, Naive Bayes, Bayesian Network and NB tree. Experiments show that considering the F-measure as evaluation metric, the proposed approach yields 1.8 to 2.4 percent performance improvement compared to other classifiers.

Increasing Face Recognition Rates Using Novel Classification Algorithms

This paper describes and discusses a set of algorithms which can improve ace recognition rates. These algorithms include adaptive K-Nearest Neighbour, daptive weighted average, reverse weighted average and exponential weighted average. ssentially, the algorithms are extensions to the basic classification algorithm sed in most face recognition research. Whereas the basic classification algorithm elects the subject with the shortest associated distance, the algorithms presented in his paper manipulate and extract information from the set of distances between a est image and the training image set in order to obtain more accurate classifications. he base system to which the algorithms are applied uses the eigenfaces technique or recognition with an adapted Viola and Jones algorithm for face extraction. Most f the algorithms proposed show a consistent improvement over the baseline test.

Improved classification with allocation method and multiple classifiers

Classification is the most used supervized machine learning method. As each of the many existing classification algorithms can perform poorly on some data, different attempts have arisen to improve the original algorithms by combining them. Some of the best know results are produced by ensemble methods, like bagging or boosting. We developed a new ensemble method called allocation. Allocation method uses the allocator, an algorithm that separates the data instances based on anomaly detection and allocates them to one of the micro classifiers, built with the existing classification algorithms on a subset of training data. The outputs of micro classifiers are then fused together into one final classification. Our goal was to improve the results of original classifiers with this new allocation method and to compare the classification results with existing ensemble methods. The allocation method was tested on 30 benchmark datasets and was used with six well known basic classification algorithms (J48, NaiveBayes, IBk, SMO, OneR and NBTree). The obtained results were compared to those of the basic classifiers as well as other ensemble methods (bagging, MultiBoost and AdaBoost). Results show that our allocation method is superior to basic classifiers and also to tested ensembles in classification accuracy and f-score. The conducted statistical analysis, when all of the used classification algorithms are considered, confirmed that our allocation method performs significantly better both in classification accuracy and f-score. Although the differences are not significant for each of the used basic classifier alone, the allocation method achieved the biggest improvements on all six basic classification algorithms. In this manner, allocation method proved to be a competitive ensemble method for classification that can be used with various classification algorithms and can possibly outperform other ensembles on different types of data.

Hierarchical Categorization of Open Source Software by Online Profiles

The large amounts of freely available open source software over the Internet are fundamentally changing the traditional paradigms of software development. Efficient categorization of the massive projects for retrieving relevant software is of vital importance for Internet-based software development such as solution searching, best practices learning and so on. Many previous works have been conducted on software categorization by mining source code or byte code, but were verified on only relatively small collections of projects with coarse-grained categories or clusters. However, Internet-based software development requires finer-grained, more scalable and language-independent categorization approaches. In this paper, we propose a novel approach to hierarchically categorize software projects based on their online profiles. We design a SVM-based categorization framework and adopt a weighted combination strategy to aggregate different types of profile attributes from multiple repositories. Different basic classification algorithms and feature selection techniques are employed and compared. Extensive experiments are carried out on more than 21,000 projects across five repositories. The results show that our approach achieves significant improvements by using weighted combination. Compared to the previous work, our approach presents competitive results with more finer-grained and multi-layered category hierarchy with more than 120 categories. Unlike approaches that use source code or byte code, our approach is more effective for large-scale and language-independent software categorization. In addition, experiments suggest that hierarchical categorization combined with general keyword-based searching improves the retrieval efficiency and accuracy.

LVQ-SMOTE – Learning Vector Quantization based Synthetic Minority Over–sampling Technique for biomedical data

BackgroundOver-sampling methods based on Synthetic Minority Over-sampling Technique (SMOTE) have been proposed for classification problems of imbalanced biomedical data. However, the existing over-sampling methods achieve slightly better or sometimes worse result than the simplest SMOTE. In order to improve the effectiveness of SMOTE, this paper presents a novel over-sampling method using codebooks obtained by the learning vector quantization. In general, even when an existing SMOTE applied to a biomedical dataset, its empty feature space is still so huge that most classification algorithms would not perform well on estimating borderlines between classes. To tackle this problem, our over-sampling method generates synthetic samples which occupy more feature space than the other SMOTE algorithms. Briefly saying, our over-sampling method enables to generate useful synthetic samples by referring to actual samples taken from real-world datasets.ResultsExperiments on eight real-world imbalanced datasets demonstrate that our proposed over-sampling method performs better than the simplest SMOTE on four of five standard classification algorithms. Moreover, it is seen that the performance of our method increases if the latest SMOTE called MWMOTE is used in our algorithm. Experiments on datasets for β-turn types prediction show some important patterns that have not been seen in previous analyses.ConclusionsThe proposed over-sampling method generates useful synthetic samples for the classification of imbalanced biomedical data. Besides, the proposed over-sampling method is basically compatible with basic classification algorithms and the existing over-sampling methods.