Binary Classification Algorithm Research Articles

Auto Encoder Fixed-Target Training Features Extraction Approach for Binary Classification Problems

The main issues with machine learning-based feature extraction techniques are the requirement of extensive domain-level knowledge, experience, and the need to be supported by large amounts of data that are sometimes not available. Moreover, it is often difficult to apply domain-level knowledge to extract the necessary features for building a machine-learning classifier. Therefore, it is significantly important to find and develop feature extraction techniques that depend mainly on the training data and don’t require or depend on domain-level knowledge and experience. To address these issues for binary classification problems, a novel feature extraction approach, AE-FT(Fixed Target) for extracting common features using a Deep Belief Network (DBN)-based Autoencoder (AE) is proposed in this paper. In this approach, common features are extracted by a DBN trained on a dataset sample’s binary using the Fixed Target training approach.
 The proposed common features extraction approach is tested and evaluated on two different data sets. For each dataset, the extracted features are used to train seven of the common machine learning binary classification algorithms and compared their performances. Moreover, the number of extracted features is very small compared to other existing feature extraction methods. Therefore, the proposed common features extraction method improves the performance of the binary classification algorithms by reducing the number of features reducing laborious processes, and increasing the recognition accuracy effectively.
 The results show that the proposed common features extraction approach, without any domain-level knowledge or human expertise, provides a very good performance compared to other feature extraction techniques.

Kidney stone classification using deep learning neural network

Kidney stones are the common problem in the healthcare system. It is rapidly increasing day by day and becomes a global health crisis in worldwide. Various deep learning algorithms are used for classificationof stone in kidney area. The computer aided design approach can be used for assist doctor for finding out the stone in kidney area. For kidney transplantation and dialysis, a proper treatment is required. It is important to have reliable techniques for predicting kidney stone size atits early stages. Different machine learning (ML) algorithmsare given excellent results in predicting stone. In this paper, clinicaldata is used for predicting of stone in kidney. If data have some missing values, data unbalancing problem then machine learning algorithms assist to solve this problem in which includes data preprocessing, a technique for managing missing values, data aggregation, feature extraction and prediction of result by evaluating values. In this study, deep learning algorithm for classification of kidney stone sizes automatically on the patient’s dataset is used. A total of 1000 patient’s dataset are used for finding out kidney stone size i.e., large or small. The binary classification algorithm is used for classification of stone size. We observed that our model gives best result for classification of kidney stone image size.

Using machine learning methods to forecast the number of power outages at substations

Forecasting in the energy sector is of great importance for suppliers and for consumers. Optimum power consumption depends on many factors. Due to natural or any other external conditions, accidents are possible. In order to minimize emergency consequences, it is necessary to be prepared for possible outages in advance in order to reduce the time for their elimination and decision-making. This article considers the problem of forecasting power outages at substations. The enterprise provided a summary table of outages at substations due to natural disasters on specific days. To solve the problem, a machine learning method was chosen – binary classification. Five different algorithms were considered. The models were tested on data from the first half of 2022. The most effective algorithm for 20% of the test sample was the binary classification algorithm using generalized additive models (GAM). This algorithm is also one of the best with a sample of 50%. A model has been prepared for further use in predicting the probability of outages at the enterprise. The model can be used in other organizations; for this, it is first necessary to train the model on the data of the corresponding region.

An Oscillator-Synchronization-Based Off-Line Learning Algorithm, With On-Chip Inference on an Array of Spin Hall Nano-Oscillators

Learning algorithms based on synchronization of oscillators are currently being pursued as an alternative to standard neural-network algorithms for data classification. For such oscillator-based algorithms, while learning can be implemented off-line on a computer, on-chip inference/ testing can be implemented on an array of uniform-mode spin Hall nano-oscillators (SHNOs), leading to a scalable and energy-efficient technology. Here, we propose a modification to an existing oscillator-based off-line learning algorithm for binary classification: unlike in the previous version of the algorithm, here the difference between natural frequencies of the output oscillators is kept constant throughout the learning process. This helps in preservation of the shape of the synchronization region and leads to higher classification accuracy, as we show for binary-classification tasks using two popular data sets: Fisher's Iris and MNIST. Next, in this paper, we show how a synchronization pattern obtained after such training, using our proposed algorithm, can be implemented on an array of dipole-coupled uniform-mode SHNOs for on-chip inference. We model the SHNO system through the macro-spin model, which is computationally much less resource-intensive to simulate compared to the micromagnetic model used previously for this kind of study. We also extend our algorithm to multi-class classification and thereby discuss scaling of our system.

Quantum-inspired algorithm for direct multi-class classification

Over the last few decades, quantum machine learning has emerged as a groundbreaking discipline. Harnessing the peculiarities of quantum computation for machine learning tasks offers promising advantages. Quantum-inspired machine learning has revealed how relevant benefits for machine learning problems can be obtained using the quantum information theory even without employing quantum computers. In the recent past, experiments have demonstrated how to design an algorithm for binary classification inspired by the method of quantum state discrimination, which exhibits high performance with respect to several standard classifiers. However, a generalization of this quantum-inspired binary classifier to a multi-class scenario remains nontrivial. Typically, a simple solution in machine learning decomposes multi-class classification into a combinatorial number of binary classifications, with a concomitant increase in computational resources. In this study, we introduce a quantum-inspired classifier that avoids this problem. Inspired by quantum state discrimination, our classifier performs multi-class classification directly without using binary classifiers. We first compared the performance of the quantum-inspired multi-class classifier with eleven standard classifiers. The comparison revealed an excellent performance of the quantum-inspired classifier. Comparing these results with those obtained using the decomposition in binary classifiers shows that our method improves the accuracy and reduces the time complexity. Therefore, the quantum-inspired machine learning algorithm proposed in this work is an effective and efficient framework for multi-class classification. Finally, although these advantages can be attained without employing any quantum component in the hardware, we discuss how it is possible to implement the model in quantum hardware.

An Investigation to Detect Banking Malware Network Communication Traffic Using Machine Learning Techniques

Banking malware are malicious programs that attempt to steal confidential information, such as banking authentication credentials, from users. Zeus is one of the most widespread banking malware variants ever discovered. Since the Zeus source code was leaked, many other variants of Zeus have emerged, and tools such as anti-malware programs exist that can detect Zeus; however, these have limitations. Anti-malware programs need to be regularly updated to recognise Zeus, and the signatures or patterns can only be made available when the malware has been seen. This limits the capability of these anti-malware products because they are unable to detect unseen malware variants, and furthermore, malicious users are developing malware that seeks to evade signature-based anti-malware programs. In this paper, a methodology is proposed for detecting Zeus malware network traffic flows by using machine learning (ML) binary classification algorithms. This research explores and compares several ML algorithms to determine the algorithm best suited for this problem and then uses these algorithms to conduct further experiments to determine the minimum number of features that could be used for detecting the Zeus malware. This research also explores the suitability of these features when used to detect both older and newer versions of Zeus as well as when used to detect additional variants of the Zeus malware. This will help researchers understand which network flow features could be used for detecting Zeus and whether these features will work across multiple versions and variants of the Zeus malware.

Deep learning-based detection of aphid colonies on plants from a reconstructed Brassica image dataset

Plant pathogenic colony-forming aphids are serious pests of agricultural crops. Most aphid species develop colonies, which are several to many aphids that are functionally organized to extract food from plants. Early and efficient detection of aphid colonies enables the implementation of control measures to reduce crop damage. Aphid colonies exhibit different shapes, sizes, and numbers of individuals, and their distribution makes it hard to detect them correctly. This paper investigates machine learning-based aphid colony detection from imagery. To the best of our knowledge, it is the first study that uses artificial intelligence-based computing algorithms to detect aphid colonies from images. As such, no aphid colony image dataset is currently available publicly. To mitigate this, we have relabelled an existing insect and pest dataset and repurposed it as an aphid colony dataset (AphColDat). For labelling, first, we automatically identify the regions of interest for colonies based on the locations and distributions of aphids. A novel bounding box merging technique is proposed to generate regions potential colony boxes. These colony boxes are a collection of single or overlapping aphid boxes that co-locate together in a colony. Once the dataset is constructed, a convolutional neural network (CNN)-based binary classification algorithm is applied to the images to create AphColDat. This paper evaluates several object detecting deep learning models on the newly developed dataset. The results demonstrate mean average precisions (mAP) of 56.9%, 53.4%, 53.1%, and 48.7% respectively by Faster R-CNN, SSD, YOLOv3, and EfficientNet. In terms of average detection speed (computational time), SSD and EfficientNet are faster than Faster R-CNN and YOLOv3.

Comparing linear discriminant analysis and supervised learning algorithms for binary classification-A method comparison study.

In psychology, linear discriminant analysis (LDA) is the method of choice for two-group classification tasks based on questionnaire data. In this study, we present a comparison of LDA with several supervised learning algorithms. In particular, we examine to what extent the predictive performance of LDA relies on the multivariate normality assumption. As nonparametric alternatives, the linear support vector machine (SVM), classification and regression tree (CART), random forest (RF), probabilistic neural network (PNN), and the ensemble k conditional nearest neighbor (EkCNN) algorithms are applied. Predictive performance is determined using measures of overall performance, discrimination, and calibration, and is compared in two reference data sets as well as in a simulation study. The reference data are Likert-type data, and comprise 5 and 10 predictor variables, respectively. Simulations are based on the reference data and are done for a balanced and an unbalanced scenario in each case. In order to compare the algorithms' performance, data are simulated from multivariate distributions with differing degrees of nonnormality. Results differ depending on the specific performance measure. The main finding is that LDA is always outperformed by RF in the bimodal data with respect to overall performance. Discriminative ability of the RF algorithm is often higher compared to LDA, but its model calibration is usually worse. Still LDA mostly ranges second in cases it is outperformed by another algorithm, or the differences are only marginal. In consequence, we still recommend LDA for this type ofapplication.

A Deep Learning Approach for Detecting Stroke from Brain CT Images Using OzNet.

A brain stroke is a life-threatening medical disorder caused by the inadequate blood supply to the brain. After the stroke, the damaged area of the brain will not operate normally. As a result, early detection is crucial for more effective therapy. Computed tomography (CT) images supply a rapid diagnosis of brain stroke. However, while doctors are analyzing each brain CT image, time is running fast. This circumstance may lead to result in a delay in treatment and making errors. Therefore, we targeted the utilization of an efficient artificial intelligence algorithm in stroke detection. In this paper, we designed hybrid algorithms that include a new convolution neural networks (CNN) architecture called OzNet and various machine learning algorithms for binary classification of real brain stroke CT images. When we classified the dataset with OzNet, we acquired successful performance. However, for this target, we combined it with a minimum Redundancy Maximum Relevance (mRMR) method and Decision Tree (DT), k-Nearest Neighbors (kNN), Linear Discriminant Analysis (LDA), Naïve Bayes (NB), and Support Vector Machines (SVM). In addition, 4096 significant features were obtained from the fully connected layer of OzNet, and we reduced the dimension of features from 4096 to 250 using the mRMR method. Finally, we utilized these machine learning algorithms to classify important features. As a result, OzNet-mRMR-NB was an excellent hybrid algorithm and achieved an accuracy of 98.42% and AUC of 0.99 to detect stroke from brain CT images.

Detection of Nonexudative Macular Neovascularization on Structural OCT Images Using Vision Transformers.

A deep learning model was developed to detect nonexudative macular neovascularization (neMNV) using OCT B-scans. Retrospective review of a prospective, observational study. Normal control eyes and patients with age-related macular degeneration (AMD) with and without neMNV. Swept-source OCT angiography (SS-OCTA) imaging (PLEX Elite 9000, Carl Zeiss Meditec, Inc) was performed using the 6× 6-mm scan pattern. Individual B-scans were annotated to distinguish between drusen and the double-layer sign (DLS) associated with the neMNV. The machine learning model was tested on a dataset graded by humans, and model performance was compared with the human graders. Intersection over Union (IoU) score was measured to evaluate segmentation network performance. Area under the receiver operating characteristic curve values, sensitivity, specificity, and positive predictive value (PPV) and negative predictive value (NPV) were measured to assess the performance of the final classification performance. Chance-corrected agreement between the algorithm and the human grader determinations was measured with Cohen's kappa. A total of 251 eyes from 210 patients, including 182 eyes with DLS and 115 eyes with drusen, were used for model training. Of 125 500 B-scans, 6879 B-scans were manually annotated. A vision transformer segmentation model was built to extract DLS and drusen from B-scans. The extracted prediction masks from all B-scans in a volume were projected to an en face image, and an eye-level projection map was obtained for each eye. A binary classification algorithm was established to identify eyes with neMNV from the projection map. The algorithm achieved 82%, 90%, 79%, and 91% sensitivity, specificity, PPV, and NPV, respectively, on a separate test set of 100 eyes that were evaluated by human graders in a previous study. The area under the curve value was calculated as 0.91 (95% confidence interval, 0.85-0.98). The results of the algorithm showed excellent agreement with the senior human grader (kappa= 0.83, P < 0.001) and moderate agreement with the junior grader consensus (kappa= 0.54, P < 0.001). Our network (code is available at https://github.com/uw-biomedical-ml/double_layer_vit) was able to detect the presence of neMNV from structural B-scans alone by applying a purely transformer-based model.

Plasma image classification using cosine similarity constrained convolutional neural network

Plasma jets are widely investigated both in the laboratory and in nature. Astrophysical objects such as black holes, active galactic nuclei and young stellar objects commonly emit plasma jets in various forms. With the availability of data from plasma jet experiments resembling astrophysical plasma jets, classification of such data would potentially aid in not only investigating the underlying physics of the experiments but also the study of astrophysical jets. In this work we use deep learning to process all of the laboratory plasma images from the Caltech Spheromak Experiment spanning two decades. We found that cosine similarity can aid in feature selection, classify images through comparison of feature vector direction and be used as a loss function for the training of AlexNet for plasma image classification. We also develop a simple vector direction comparison algorithm for binary and multi-class classification. Using our algorithm we demonstrate 93 % accurate binary classification to distinguish unstable columns from stable columns and 92 % accurate five-way classification of a small, labelled data set which includes three classes corresponding to varying levels of kink instability.

Multi-Scale Remaining Useful Life Prediction Using Long Short-Term Memory

Predictive maintenance based on performance degradation is a crucial way to reduce maintenance costs and potential failures in modern complex engineering systems. Reliable remaining useful life (RUL) prediction is the main criterion for decision-making in predictive maintenance. Conventional model-based methods and data-driven approaches often fail to achieve an accurate prediction result using a single model for a complex system featuring multiple components and operational conditions, as the degradation pattern is usually nonlinear and time-varying. This paper proposes a novel multi-scale RUL prediction approach adopting the Long Short-Term Memory (LSTM) neural network. In the feature engineering phase, Pearson’s correlation coefficient is applied to extract the representative features, and an operation-based data normalisation approach is presented to deal with the cases where multiple degradation patterns are concealed in the sensor data. Then, a three-stage RUL target function is proposed, which segments the degradation process of the system into the non-degradation stage, the transition stage, and the linear degradation stage. The classification of these three stages is regarded as the small-scale RUL prediction, and it is achieved through processing sensor signals after the feature engineering using a novel LSTM-based binary classification algorithm combined with a correlation method. After that, a specific LSTM-based predictive model is built for the last two stages to produce a large-scale RUL prediction. The proposed approach is validated by comparing it with several state-of-the-art techniques based on the widely used C-MAPSS dataset. A significant improvement is achieved in RUL prediction performance in most subsets. For instance, a 40% reduction is achieved in Root Mean Square Error over the best existing method in subset FD001. Another contribution of the multi-scale RUL prediction approach is that it offers more degree of flexibility of prediction in the maintenance strategy depending on data availability and which degradation stage the system is in.

The use of machine learning methods to predict sperm quality in Holstein bulls

The aim of this study was to develop prediction models for total sperm motility, morphological abnormalities and sperm output based on 1,551 ejaculate records of 58 Holstein bulls. The data was collected from September 2019 to November 2020 in a single artificial insemination (AI) center located in Eastern Germany. Factors considered for the prediction models include barn climate conditions, semen collector, number of false mounts, libido, semen collection frequency, breed and age (10–74 months). In this study, the prediction models Lasso, Group Lasso and Gradient Boosting were evaluated. The best model for each sperm quality parameter was chosen using cross validation. The models were estimated with five algorithms for sperm motility and sperm morphology and three algorithms for the number of total sperm per ejaculate (sperm output). For sperm motility and morphology a binary classification algorithm was applied, reaching an accuracy of over 80% for all models. For sperm output, no such classification was used and the only variable selected by all three algorithms was age. Furthermore, for sperm morphology, climate variables were frequently selected. Additionally, network diagrams from Group Lasso show the interdependencies between the major variable groups influencing sperm motility and morphology. In conclusion, the implementation of such prediction tools could help AI centers to optimize management factors and stabilize bull semen production in the future.

Two class Bayes point machines in repayment prediction of low credit borrowers

Decision-making in the peer-to-peer loan market has not been studied as extensively as traditional lending mostly because of the perceived risk in dealing with low credit borrowers seeking funding alternatives. We develop a machine learning-based approach to test the viability and usefulness in peer-to-peer loan repayment predictions among low credit borrowers. This analysis provides potential benefits that could strengthen the lending market with a more reliable method of identifying applications from promising candidates with low credit. Here an experiment will be performed to measure the performance of a model used for classifying peer-to-peer loan data. The aim is to aid the repayment prediction capabilities of peer lenders when analyzing low credit applicants. A binary classification algorithm is used to build the model and applied to actual historical loan data to evaluate performance. Experiment results, visualizations, and key performance indicators are discussed in the work to influence confidence in using the method proposed.

Low-Cost Electronic Nose for Wine Variety Identification through Machine Learning Algorithms

The aroma of wine is traditionally analyzed by sensory methods or by using gas chromatography; both analytical methodologies are slow and expensive and do not allow continuous monitoring. For this reason, interest in rapid methods has increased in recent times. Electronic noses (e-noses) stand out for their high sensitivity, speed, low cost, and little or no sample preparation. They present, however, low selectivity, which requires advance analytical methods to distinguish compounds. Here, we present a low-cost e-nose device for the analysis and identification of distinct varieties of wine. Chemical analysis data are compared to e-nose data through a principal component analysis (PCA) and a k-means clustering algorithm to establish relationships between varieties of wines and the e-nose classification capability. The results show that e-nose technology found significant differences between the analyzed samples, and furthermore, classifying the samples in accordance with the chemical analysis classification. The maximal accuracy obtained was 100% using the k-means algorithm for binary classification with N = 21 samples. Thus the potential of e-nose technology was shown in the wine industry for the identification and classification of wine varieties or quality.

Proposal of a new method for learning of diesel generator sounds and detecting abnormal sounds using an unsupervised deep learning algorithm

This study is to find a method to learn engine sound after the start-up of a diesel generator installed in nuclear power plant with an unsupervised deep learning algorithm (CNN autoencoder) and a new method to predict the failure of a diesel generator using it. In order to learn the sound of a diesel generator with a deep learning algorithm, sound data recorded before and after the start-up of two diesel generators was used. The sound data of 20 min and 2 h were cut into 7 s, and the split sound was converted into a spectrogram image. 1200 and 7200 spectrogram images were created from sound data of 20 min and 2 h, respectively. Using two different deep learning algorithms (CNN autoencoder and binary classification), it was investigated whether the diesel generator post-start sounds were learned as normal. It was possible to accurately determine the post-start sounds as normal and the pre-start sounds as abnormal. It was also confirmed that the deep learning algorithm could detect the virtual abnormal sounds created by mixing the unusual sounds with the post-start sounds. This study showed that the unsupervised anomaly detection algorithm has a good accuracy increased about 3% with comparing to the binary classification algorithm.

A Machine Learning in Binary and Multiclassification Results on Imbalanced Heart Disease Data Stream

In medical filed, predicting the occurrence of heart diseases is a significant piece of work. Millions of healthcare-related complexities that have remained unsolved up until now can be greatly simplified with the help of machine learning. The proposed study is concerned with the cardiac disease diagnosis decision support system. An OpenML repository data stream with 1 million instances of heart disease and 14 features is used for this study. After applying to preprocess and feature engineering techniques, machine learning approaches like random forest, decision trees, gradient boosted trees, linear support vector classifier, logistic regression, one-vs-rest, and multilayer perceptron are used to perform binary and multiclassification on the data stream. When combined with the Max Abs Scaler technique, the multilayer perceptron performed satisfactorily in both binary (Accuracy 94.8%) and multiclassification (accuracy 88.2%). Compared to the other binary classification algorithms, the GBT delivered the right outcome (accuracy of 95.8%). Multilayer perceptrons, however, did well in multiple classifications. Techniques such as oversampling and undersampling have a negative impact on disease prediction. Machine learning methods like multilayer perceptrons and ensembles can be helpful for diagnosing cardiac conditions. For this kind of unbalanced data stream, sampling techniques like oversampling and undersampling are not practical.

Predicting subscription renewal using binary classification in World of Warcraft

Multiplayer online video games in the past decade have become one of the world's most popular gaming formats. Analyzing player data can provide insights into player behavior, such as how long players continue to play or where they stop playing. This information can help developers to come up with solutions to retain their players. We have compared binary classification algorithms on different time frames of player history data along with level and guild from World of Warcraft – a previously popular massively multiplayer online role-playing game (MMORPG), to assess the probability of subscription renewal. Our findings suggest that random forest and decision tree algorithms give the highest accuracy in all cases (more than 90 %). This was achieved using level and guild as parameters along with a combination of active days and total playing time taken from different time frames in the past month along.

Evaluation of College English Teaching Quality Based on Improved BT-SVM Algorithm.

With the development of teaching evaluation program, colleges and universities have reformed according to the actual situation of the school. With the development of evaluation activities, many universities are eager to establish their own teaching quality evaluation system, so as to pre-evaluate the teaching quality of schools. SVM is one of the most widely used machine learning algorithms that enables efficient statistical learning with a very limited number of samples. Considering the excellent learning performance of SVM, it is very suitable for the teaching quality evaluation system. In this paper, we optimize the existing multiple classification algorithm for binary trees and propose a new method. Learning the popular teaching quality evaluation system in colleges and universities, the binary tree support vector machine classification algorithm, and design comparison experiment, the experimental results show that the evaluation model proposed in this paper has strong generalization ability and higher classification accuracy and better classification efficiency.

Survival prediction models: an introduction to discrete-time modeling

BackgroundPrediction models for time-to-event outcomes are commonly used in biomedical research to obtain subject-specific probabilities that aid in making important clinical care decisions. There are several regression and machine learning methods for building these models that have been designed or modified to account for the censoring that occurs in time-to-event data. Discrete-time survival models, which have often been overlooked in the literature, provide an alternative approach for predictive modeling in the presence of censoring with limited loss in predictive accuracy. These models can take advantage of the range of nonparametric machine learning classification algorithms and their available software to predict survival outcomes.MethodsDiscrete-time survival models are applied to a person-period data set to predict the hazard of experiencing the failure event in pre-specified time intervals. This framework allows for any binary classification method to be applied to predict these conditional survival probabilities. Using time-dependent performance metrics that account for censoring, we compare the predictions from parametric and machine learning classification approaches applied within the discrete time-to-event framework to those from continuous-time survival prediction models. We outline the process for training and validating discrete-time prediction models, and demonstrate its application using the open-source R statistical programming environment.ResultsUsing publicly available data sets, we show that some discrete-time prediction models achieve better prediction performance than the continuous-time Cox proportional hazards model. Random survival forests, a machine learning algorithm adapted to survival data, also had improved performance compared to the Cox model, but was sometimes outperformed by the discrete-time approaches. In comparing the binary classification methods in the discrete time-to-event framework, the relative performance of the different methods varied depending on the data set.ConclusionsWe present a guide for developing survival prediction models using discrete-time methods and assessing their predictive performance with the aim of encouraging their use in medical research settings. These methods can be applied to data sets that have continuous time-to-event outcomes and multiple clinical predictors. They can also be extended to accommodate new binary classification algorithms as they become available. We provide R code for fitting discrete-time survival prediction models in a github repository.