Binary Classification Problem Research Articles

A conditional branch predictor based on weightless neural networks

Conditional branch prediction allows the speculative fetching and execution of instructions before knowing the direction of conditional statements. As in other areas, machine learning techniques are a promising approach to building branch predictors, e.g., the Perceptron predictor. However, those traditional solutions demand large input sizes, which impose a considerable area overhead. We propose a conditional branch predictor based on the WiSARD (Wilkie, Stoneham, and Aleksander’s Recognition Device) weightless neural network model. The WiSARD-based predictor implements one-shot online training designed to address branch prediction as a binary classification problem. We compare the WiSARD-based predictor with two state-of-the-art predictors: TAGE-SC-L (TAgged GEometric-Statistical Corrector-Loop) and the Multiperspective Perceptron. Our experimental evaluation shows that our proposed predictor, with a smaller input size, outperforms the perceptron-based predictor by about 0.09% and achieves similar accuracy to that of TAGE-SC-L. In addition, we perform an experimental sensitivity analysis to find the best predictor for each dataset, and based on these results, we designed new specialized predictors using a particular parameter composition for each dataset. The results show that the specialized WiSARD-based predictor outperforms the state-of-the-art by more than 2.3% in the best case. Furthermore, through the implementation of specialized predictor classifiers, we discovered that utilizing 90% of the specialized predictor for a specific dataset yielded comparable performance to the corresponding specialized predictor.

A linear adjustment-based approach to posterior drift in transfer learning.

We present new models and methods for the posterior drift problem where the regression function in the target domain is modelled as a linear adjustment, on an appropriate scale, of that in the source domain, and study the theoretical properties of our proposed estimators in the binary classification problem. The core idea of our model inherits the simplicity and the usefulness of generalized linear models and accelerated failure time models from the classical statistics literature. Our approach is shown to be flexible and applicable in a variety of statistical settings, and can be adopted for transfer learning problems in various domains including epidemiology, genetics and biomedicine. As concrete applications, we illustrate the power of our approach (i) through mortality prediction for British Asians by borrowing strength from similar data from the larger pool of British Caucasians, using the UK Biobank data, and (ii) in overcoming a spurious correlation present in the source domain of the Waterbirds dataset.

Ensemble LDA via the modified Cholesky decomposition

A binary classification problem in the high-dimensional settings is studied via the ensemble learning with each base classifier constructed from the linear discriminant analysis (LDA), and these base classifiers are integrated by the weighted voting. The precision matrix in the LDA rule is estimated by the modified Cholesky decomposition (MCD), which is able to provide us with a set of precision estimates by considering multiple variable orderings, and hence yield a group of different LDA classifiers. Such available LDA classifiers are then integrated to improve the classification performance. The simulation and the application studies are conducted to demonstrate the merits of the proposed method.

Solving nonsmooth interval optimization problems based on interval-valued symmetric invexity

This paper focuses on a nonsmooth nonconvex interval-valued optimization problem. For this, we propose interval-valued symmetric invexity, interval-valued symmetric pseudo-invexity and interval-valued symmetric quasi-invexity in terms of the symmetric gH-differentiable interval-valued functions. Some important properties of these generalized convexities are also discussed. By utilizing these new concepts, we establish sufficient Karush–Kuhn–Tucker conditions for the considered problem. Further, the Wolfe and Mond–Weir type dual problems are associated and weak, strong and strict converse duality results have been derived. Finally, we apply the developed theory to a binary classification problem of interval data by support vector machine.

Prediction Capability of Geomagnetic Events from Solar Wind Data Using Neural Networks

Multiple neural network architectures, with different structural composition and complexity, are implemented in this study with the aim of providing multi-hour-ahead warnings of severe geomagnetic disturbances, based on in situ measurements of the solar wind plasma and magnetic field acquired at the Lagrangian point L1. First, a statistical analysis of the interplanetary data was performed to point out which are the most relevant parameters to be provided as input to the neural networks, and a preprocessing of the data set was implemented to face its heavy imbalance (the Earth’s magnetosphere is in fact mostly at rest). Then, neural networks were tested to evaluate their performance. It turned out that, in a binary classification problem, recurrent approaches are best at predicting critical events both 1 and 8 hr in advance, achieving a balanced accuracy of 94% and 70%, respectively. Finally, in an attempt at multistep prediction of the criticality of future geomagnetic events from 1–8 hr ahead, more complex neural networks, built by merging the different types of basic convolutional and recurrent architectures, have been shown to outperform single-step and state-of-the-art approaches with a balanced accuracy of at least 70%. Interestingly, the accuracy peaks at 4 hr, corresponding to the waiting time between the detection of solar drivers at L1 and the onset of the geomagnetic storm (as previously obtained by statistical investigations), suggesting that on average this is the time the Earth’s magnetosphere takes to react to the solar event.

Malicious webshell family dataset for webshell multi-classification research

Malicious webshells currently present tremendous threats to cloud security. Most relevant studies and open webshell datasets consider malicious webshell defense as a binary classification problem, that is, identifying whether a webshell is malicious or benign. However, a fine-grained multi-classification is urgently needed to enable precise responses and active defenses on malicious webshell threats. This paper introduces a malicious webshell family dataset named MWF to facilitate webshell multi-classification researches. This dataset contains 1359 malicious webshell samples originally obtained from the cloud servers of Alibaba Cloud. Each of them is provided with a family label. The samples of the same family generally present similar characteristics or behaviors. The dataset has a total of 78 families and 22 outliers. Moreover, this paper introduces the human–machine collaboration process that is adopted to remove benign or duplicate samples, address privacy issues, and determine the family of each sample. This paper also compares the distinguished features of the MWF dataset with previous datasets and summarizes the potential applied areas in cloud security and generalized sequence, graph, and tree data analytics and visualization.

Application of quantum machine learning using quantum kernel algorithms on multiclass neuron M-type classification

The functional characterization of different neuronal types has been a longstanding and crucial challenge. With the advent of physical quantum computers, it has become possible to apply quantum machine learning algorithms to translate theoretical research into practical solutions. Previous studies have shown the advantages of quantum algorithms on artificially generated datasets, and initial experiments with small binary classification problems have yielded comparable outcomes to classical algorithms. However, it is essential to investigate the potential quantum advantage using real-world data. To the best of our knowledge, this study is the first to propose the utilization of quantum systems to classify neuron morphologies, thereby enhancing our understanding of the performance of automatic multiclass neuron classification using quantum kernel methods. We examined the influence of feature engineering on classification accuracy and found that quantum kernel methods achieved similar performance to classical methods, with certain advantages observed in various configurations.

Positive unlabeled learning with tensor networks

Positive unlabeled learning is a binary classification problem with positive and unlabeled data. It is common in domains where negative labels are costly or impossible to obtain, e.g., medicine and personalized advertising. Most approaches to positive unlabeled learning apply to specific data types (e.g., images, categorical data) and can not generate new positive and negative samples. This work introduces a feature-space distance-based tensor network approach to the positive unlabeled learning problem. The presented method is not domain specific and significantly improves the state-of-the-art results on the MNIST image and 15 categorical/mixed datasets. The trained tensor network model is also a generative model and enables the generation of new positive and negative instances.

An unsupervised neural network for graphical health index construction and residual life prediction

To better characterize the health status and performing remaining useful life prediction, a composite health index is developed through the fusion of multi-channel signals. However, most of the existing literature limits the data fusion to be linear, which implies that the underlying degradation pattern must follow a linear form. This strong prerequisite of these approaches undermines the effectiveness of existing techniques for capturing the potential nonlinear nature of degradation process. In order to overcome this limitation as well as to improve the predictability, this paper proposes a nonlinear health index construction method achieving by an unsupervised neural network. Specifically, a neural network structure is introduced to approximate the highly nonlinear relationship between signals and health status. Furthermore, we consider the remaining useful life prediction as a binary classification problem, and then propose a maximal classification margin constraint, which is integrated with the monotonicity and minimal variability at the failure time to formulate the novel loss function. To estimate the model parameter, we developed a customized adaptive moment estimation algorithm (Adam). The comprehensive case study is performed based on the benchmark C-MAPSS dataset. As reported in the experiment, the constructed health index can better characterize the underlying degradation process.

Differentiating signal from artefacts in cosmic ray detection: Applying Siamese spiking neural networks to CREDO experimental data

The Cosmic Ray Extremely Distributed Observatory (CREDO) is an international research consortium aimed at observing high energy cosmic ray particles. The associated Android/iOS application enables the registration of muons with smartphone devices. The ubiquity of the CREDO infrastructure entails virtually no control over the detectors’ working conditions. In order to tag artefacts appearing in the CREDO database, we propose a Siamese spiking neural network (SNN) model, trained by optimizing Earth Mover’s Distance (EMD) between spike train outputs of the SNN. We first test the feasibility of our approach by training models on MNIST images converted into the spiking domain with novel conversion schemes. Then, on a binary classification problem of signal/artefact discrimination on CREDO images our model has achieved a class-balanced accuracy of 96.35%, close to the existing non-spiking solutions. Notably, the model has shown adaptability to input data properties in terms of the spiking network activity sparsity and prediction latency.

Hierarchical confusion matrix for classification performance evaluation

Abstract This study proposes the novel concept of hierarchical confusion matrix, opening the door for popular confusion-matrix-based (flat) evaluation measures from binary classification problems, while considering the peculiarities of hierarchical classification problems. The concept is developed to a generalised form and proven its applicability to all types of hierarchical classification problems including directed acyclic graphs, multi-path labelling, and non-mandatory leaf-node prediction. Finally, measures based on the novel confusion matrix are used for three real-world hierarchical classification applications and compared to established evaluation measures. The results, the conformity with important attributes of hierarchical classification schemes and its broad applicability justify its recommendation.

PKA2-Net: Prior Knowledge-Based Active Attention Network for Accurate Pneumonia Diagnosis on Chest X-Ray Images.

To accurately diagnose pneumonia patients on a limited annotated chest X-ray image dataset, a prior knowledge-based active attention network (PKA2-Net1) was constructed. The PKA2-Net uses improved ResNet as the backbone network and consists of residual blocks, novel subject enhancement and background suppression (SEBS) blocks and candidate template generators, where template generators are designed to generate candidate templates for characterizing the importance of different spatial locations in feature maps. The core of PKA2-Net is SEBS block, which is proposed based on the prior knowledge that highlighting distinctive features and suppressing irrelevant features can improve the recognition effect. The purpose of SEBS block is to generate active attention features without any high-level features and enhance the ability of the model to localize lung lesions. In SEBS block, first, a series of candidate templates T with different spatial energy distributions are generated and the controllability of the energy distribution in T enables active attention features to maintain the continuity and integrity of the feature space distributions. Second, Top-n templates are selected from T according to certain learning rules, which are then operated by a convolution layer for generating supervision information that can guide the inputs of SEBS block to form active attention features. We evaluated the PKA2-Net on the binary classification problem of identifying pneumonia and healthy controls on a dataset containing 5856 chest X-ray images (ChestXRay2017), the results showed that our method can achieve 97.63% accuracy and 0.9872 sensitivity.

Decision Tree Learning Based Feature Selection and Evaluation for Image Classification

Abstract: The problem statement focuses on feature evaluation and selection for image classification using decision tree learning. The objective is to identify the most significant features in an image dataset and train a decision tree classifier using these selected features. The accuracy of an image classifier heavily relies on the quality and relevance of the features used to represent the images. Hence, it is crucial to identify the most important features and eliminate the irrelevant ones to enhance the classifier's accuracy. To implement this approach, we can utilize scikit-learn, a popular machine learning library in Python. The solution must involve training a decision tree classifier on the dataset and extracting feature importances, selecting the top features using modules from sklearn like “SelectFromModel”, and also performing hyperparameter tuning using “GridSearchCV” and training a new decision tree classifier on the selected features with the best hyperparameters. Decision trees are a popular machine learning algorithm that uses a tree-like model of decisions and their possible consequences. By training a decision tree classifier on an image dataset and extracting feature importances, it is possible to identify the most important features and select them for use in a new decision tree classifier that can improve classification accuracy. It is important to note that decision tree learning is a versatile machine learning algorithm that can handle both binary and multiclass classification problems. Additionally, it is advantageous for feature evaluation and selection in image classification tasks. By identifying the most relevant features, this approach can enhance the accuracy of the classifier and reduce computational complexity, making it suitable for large datasets. By following this outlined approach, you can create a project that addresses feature evaluation, selection, and classification accuracy improvement using decision tree learning in the context of image classification

Detecting Level of Depression from Social Media Posts for the Low-resource Bengali Language

Depression is a mental illness that suffers people in their thoughts and daily activities. In extreme cases, sometimes it leads to self-destruction or commit to suicide. Besides an individual, depression harms the victim's family, society, and working environment. Therefore, before physiological treatment, it is essential to identify depressed people first. As various social media platforms like Facebook overwhelm our everyday life, depressed people share their personal feelings and opinions through these platforms by sending posts or comments. We have detected many research work that experiment on those text messages in English and other highly-resourced languages. Limited works we have identified in low-resource languages like Bengali. In addition, most of these works deal with a binary classification problem. We classify the Bengali depression text into four classes: non-depressive, mild, moderate, and severe in this investigation. At first, we developed a depression dataset of 2,598 entries. Then, we apply pre-processing tasks, feature selection techniques, and three types of machine learning (ML) models: classical ML, deep-learning (DL), and transformer-based pre-trained models. The XLM-RoBERTa-based pre-trained model outperforms with 61.11% F1-score and 60.89% accuracy the existing works for the four levels of the depression-class classification problem. Our proposed machine learning-based automatic detection system can recognize the various stages of depression, from low to high. It may assist the psychologist or others in providing level-wise counseling to depressed people to return to their ordinary life.

On the Complexity of PAC Learning in Hilbert Spaces

We study the problem of binary classification from the point of view of learning convex polyhedra in Hilbert spaces, to which one can reduce any binary classification problem. The problem of learning convex polyhedra in finite-dimensional spaces is sufficiently well studied in the literature. We generalize this problem to that in a Hilbert space and propose an algorithm for learning a polyhedron which correctly classifies at least 1 − ε of the distribution, with a probability of at least 1 − δ, where ε and δ are given parameters. Also, as a corollary, we improve some previous bounds for polyhedral classification in finite-dimensional spaces.

FedABC: Targeting Fair Competition in Personalized Federated Learning

Federated learning aims to collaboratively train models without accessing their client's local private data. The data may be Non-IID for different clients and thus resulting in poor performance. Recently, personalized federated learning (PFL) has achieved great success in handling Non-IID data by enforcing regularization in local optimization or improving the model aggregation scheme on the server. However, most of the PFL approaches do not take into account the unfair competition issue caused by the imbalanced data distribution and lack of positive samples for some classes in each client. To address this issue, we propose a novel and generic PFL framework termed Federated Averaging via Binary Classification, dubbed FedABC. In particular, we adopt the ``one-vs-all'' training strategy in each client to alleviate the unfair competition between classes by constructing a personalized binary classification problem for each class. This may aggravate the class imbalance challenge and thus a novel personalized binary classification loss that incorporates both the under-sampling and hard sample mining strategies is designed. Extensive experiments are conducted on two popular datasets under different settings, and the results demonstrate that our FedABC can significantly outperform the existing counterparts.

Rainfall Prediction using Decision Tree: A Case Study of CST, Phuntsholing

Rainfall Prediction using Decision Tree: A Case Study of CST, Phuntsholing

LC-MS/MS-Based Absolute Quantitation of Hemoglobin Subunits from Dried Blood Spots Reveals Novel Biomarkers for α-Thalassemia Silent Carriers.

Identification of α-thalassemia silent carriers is challenging with conventional phenotype-based screening methods. A liquid chromatography tandem mass spectrometry (LC-MS/MS)-based approach may offer novel biomarkers to address this conundrum. In this study, we collected dried blood spot samples from individuals with three α-thalassemia subtypes for biomarker discovery and validation. We observed differential expression patterns of hemoglobin subunits among various α-thalassemia subtypes and normal controls through proteomic profiling of 51 samples in the discovery phase. Then, we developed and optimized a multiple reaction monitoring (MRM) assay to measure all detectable hemoglobin subunits. The validation phase was conducted in a cohort of 462 samples. Among the measured hemoglobin subunits, subunit μ was significantly upregulated in all the α-thalassemia groups with distinct fold changes. The hemoglobin subunit μ exhibits great potential as a novel biomarker for α-thalassemia, especially for silent α-thalassemia. We constructed predictive models based on the concentrations of hemoglobin subunits and their ratios to classify the various subtypes of α-thalassemia. In the binary classification problems of silent α-thalassemia vs normal, non-deletional α-thalassemia vs normal, and deletional α-thalassemia vs normal, the best performance of the models achieved average ROCAUCs of 0.9505, 0.9430, and 0.9976 in the cross-validation, respectively. In the multiclass model, the best performance achieved an average ROCAUC of 0.9290 in cross-validation. The performance of our MRM assay and models demonstrated that the hemoglobin subunit μ would play a vital role in screening silent α-thalassemia in clinical practice.

Feature selection for binary classification based on class labeling, SOM, and hierarchical clustering

Feature selection plays an important role in algorithms for processing high-dimensional data. Traditional pattern classification and information theory methods are widely applied to feature selection methods. However, traditional pattern classification methods such as Fisher Score, Laplacian Score, and relief use class labels inadequately. Previous information theory based feature selection methods such as MIFS ignore the intra-class to tight inter-class to sparse property of the samples. To address these problems, a feature selection algorithm for the binary classification problem is proposed, which is based on class label transformation using self-organizing mapping neural network (SOM) and cohesive hierarchical clustering. The algorithm first converts class labels without numerical meaning into numerical values that can participate in operations and retain classification information through class label mapping, and constitutes a two-dimensional vector from it and the attribute values to be judged. Then, these two-dimensional vectors are clustered by using SOM neural network and hierarchical clustering. Finally, evaluation function value is calculated, that is closely related to intra-cluster to tightness, inter-cluster separation, and division accuracy after clustering, and is used to evaluate the ability of alternative attributes to distinguish between classes. It is experimentally verified that the algorithm is robust and can effectively screen attributes with strong classification ability and improve the prediction performance of the classifier.

Loan Default Prediction: A Complete Revision of LendingClub

The study aims to determine a credit default prediction model using data from LendingClub. The model estimates the effect of the influential variables on the prediction process of paid and unpaid loans. We implemented the random forest algorithm to identify the variables with the most significant influence on payment or default, addressing nine predictors related to the borrower's credit and payment background. Results confirm that the model’s performance generates a F1 Macro Score that accomplishes 90% in accuracy for the evaluation sample. Contributions of this study include using the complete dataset of the entire operation of LendingClub available, to obtain transcendental variables for the classification and prediction task, which can be helpful to estimate the default in the person-to-person loan market. We can draw two important conclusions, first we confirm the Random Forest algorithm's capacity to predict binary classification problems based on performance metrics obtained and second, we denote the influence of traditional credit scoring variables on default prediction problems.