General Classification Problems Research Articles

Unsupervised Domain-Adaptive Object Detection via Localization Regression Alignment.

Unsupervised domain-adaptive object detection uses labeled source domain data and unlabeled target domain data to alleviate the domain shift and reduce the dependence on the target domain data labels. For object detection, the features responsible for classification and localization are different. However, the existing methods basically only consider classification alignment, which is not conducive to cross-domain localization. To address this issue, in this article, we focus on the alignment of localization regression in domain-adaptive object detection and propose a novel localization regression alignment (LRA) method. The idea is that the domain-adaptive localization regression problem can be transformed into a general domain-adaptive classification problem first, and then adversarial learning is applied to the converted classification problem. Specifically, LRA first discretizes the continuous regression space, and the discrete regression intervals are treated as bins. Then, a novel binwise alignment (BA) strategy is proposed through adversarial learning. BA can further contribute to the overall cross-domain feature alignment for object detection. Extensive experiments are conducted on different detectors in various scenarios, and the state-of-the-art performance is achieved; these results demonstrate the effectiveness of our method. The code will be available at: https://github.com/zqpiao/LRA.

BranchLabelNet: Anatomical Human Airway Labeling Approach using a Dividing-and-Grouping Multi-Label Classification.

Anatomical airway labeling is crucial for precisely identifying airways displaying symptoms such as constriction, increased wall thickness, and modified branching patterns, facilitating the diagnosis and treatment of pulmonary ailments. This study introduces an innovative airway labeling methodology, BranchLabelNet, which accounts for the fractal nature of airways and inherent hierarchical branch nomenclature. In developing this methodology, branch-related parameters, including position vectors, generation levels, branch lengths, areas, perimeters, and more, are extracted from a dataset of 1000 chest computed tomography (CT) images. To effectively manage this intricate branch data, we employ an n-ary tree structure that captures the complicated relationships within the airway tree. Subsequently, we employ a divide-and-group deep learning approach for multi-label classification, streamlining the anatomical airway branch labeling process. Additionally, we address the challenge of class imbalance in the dataset by incorporating the Tomek Links algorithm to maintain model reliability and accuracy. Our proposed airway labeling method provides robust branch designations and achieves an impressive average classification accuracy of 95.94% across fivefold cross-validation. This approach is adaptable for addressing similar complexities in general multi-label classification problems within biomedical systems.

Neural network classification of eigenmodes in the magnetohydrodynamic spectroscopy code Legolas

A neural network is employed to address a non-binary classification problem of plasma instabilities in astrophysical jets, calculated with the Legolas code. The trained models exhibit reliable performance in the identification of the two instability types supported by these jets. We also discuss the generation of artificial data and refinement of predictions in general eigenfunction classification problems.

New Bounds on the Accuracy of Majority Voting for Multiclass Classification.

Majority voting is a simple mathematical function that returns the most frequently occurring value within a given set. As a popular decision fusion technique (DFT), the majority voting function (MVF) finds applications in resolving conflicts, where several independent voters report their opinions on a classification problem. Despite its importance and its various applications in ensemble learning, data crowdsourcing, remote sensing, and data oracles for blockchains, the accuracy of the MVF for the general multiclass classification problem has remained unknown. In this article, we derive a new upper bound on the accuracy of the MVF for the multiclass classification problem. More specifically, we show that under certain conditions, the error rate of the MVF exponentially decays toward 0 as the number of independent voters increases. Conversely, the error rate of the MVF exponentially grows with the number of voters if these conditions are not met. We first explore the problem for voters with independent and identically distributed (i.i.d.) outputs, where we assume that, given the true classification of the data point, every voter follows the same conditional probability distribution of voting for different classes. Next, we extend our results to encompass independent but nonidentically distributed votes. Using the derived results, we then provide a discussion on the accuracy of the truth discovery algorithms. We show that in the best-case scenarios, truth discovery algorithms operate as an amplified MVF and thereby achieve a small error rate only when the MVF achieves a small error rate too, and vice versa, achieve a large error rate when the MVF also achieves a large error rate. However, in the worst case scenario, the truth discovery algorithms may exhibit a significantly higher error rate than the MVF. Finally, we confirm our theoretical results using numerical simulations.

Multi-Time-Scale Shapelet-Based Feature Extraction for Non-Intrusive Load Monitoring

The performance of non-intrusive load monitoring (NILM) system is closely related to the uniqueness and comprehensiveness of the load features extracted from the equipment. However, existing feature extraction methods for NILM are difficult to extract interpretable intuitive features representing the uniqueness of load, and lack of comprehensive attention to the features on different time scales. Therefore, this paper proposes a multi-time-scale shapelet based feature extraction framework. First, shapelet, the most unique segment of a sequence, which can distinguish it from other sequences is introduced as a feature and multi-time-scale current shapelets are proposed to extract comprehensive characteristics of electric appliances and offer intuitive interpretability. Meanwhile, a shapelet transform is constructed, so that the essence of shapelet based NILM is transformed from sequence matching to a general classification problem. Next, an ensemble bagging classifier is proposed on multi-time scales to effectively balance the classification errors on each time scale and enhance the performance of NILM system. In addition, this framework can support small sample training to facilitate rapid deployment of NILM applications at the initial practical stage. The results show that this framework can effectively focus on the uniqueness of load on different time scales, provide interpretability for NILM, and achieve good classification effects.

A cloud model-based interval-valued evidence fusion method and its application in fault diagnosis

In fault diagnosis methods based on evidence fusion, a piece of evidence is usually extracted from the fault feature information, which uses the single-valued belief degrees (SVB) to measure the likelihoods of the different fault modes happening. However, SVB seems rough and incomplete for modeling fault information with uncertainty. Therefore, an interval-valued evidence (IVE) fusion method is proposed via cloud model. Compared to the SVB structure, the IVE with the interval-valued belief structure contains more useful fault information. Firstly, the nominal normal cloud model (NCM) about each fault mode and the test normal cloud model (TNCM) are established through the statistical analysis of the historical and testing fault sample data, respectively. TNCMs are matched with each NCM to obtain the corresponding IVE. Then, a novel IVE reasoning rule is defined to combine all IVEs coming from different fault features. Finally, in the fault diagnosis experiments of motor rotors, the proposed method is verified to provide more accurate fault decision results compared to the traditional evidence fusion method. Furthermore, the proposed method is extended to solve general data classification problems with benchmark datasets of University of California Irvine (UCI), showing it has better classification ability compared to some classification methods.

Few-shot classification with multisemantic information fusion network

Metric-based methods aim to predict class labels by computing the similarity between samples using distance functions, which is the mainstream approach to few-shot learning. However, the limited representational space of feature vectors and appearance variations among congenetic samples still present challenges. We propose a Multisemantic Information Fusion Network (MIFN) to address these problems. A Lower-level Feature generator (LF-generator), which is an unsupervised module, adaptively activates high-response regions of objects to introduce discriminative semantic details. Meanwhile, a Higher-level Feature extractor (HF-extractor) learns global semantic information with human cognition to minimise the impact of appearance variations. We integrate the coarse outputs of these two modules, which complement each other to jointly promote more precise predictions. Furthermore, considering the importance of prototypes, we redefine the sampling strategy of the triplet loss and utilise it as an auxiliary loss to sharpen the decision boundary at the prototype level, facilitating subsequent classification. Our experimental results demonstrate the competitiveness of our approach in both general few-shot classification (mini-ImageNet and tiered-ImageNet) and cross-domain problems (CUB, Caltech-101, Stanford-dogs, and Stanford-cars) with minimal bells and whistles.

A Review of the F-Measure: Its History, Properties, Criticism, and Alternatives

Methods to classify objects into two or more classes are at the core of various disciplines. When a set of objects with their true classes is available, a supervised classifier can be trained and employed to decide if, for example, a new patient has cancer or not. The choice of performance measure is critical in deciding which supervised method to use in any particular classification problem. Different measures can lead to very different choices, so the measure should match the objectives. Many performance measures have been developed, and one of them is the F-measure, the harmonic mean of precision and recall. Originally proposed in information retrieval, the F-measure has gained increasing interest in the context of classification. However, the rationale underlying this measure appears weak, and unlike other measures, it does not have a representational meaning. The use of the harmonic mean also has little theoretical justification. The F-measure also stresses one class, which seems inappropriate for general classification problems. We provide a history of the F-measure and its use in computational disciplines, describe its properties, and discuss criticism about the F-Measure. We conclude with alternatives to the F-measure, and recommendations of how to use it effectively.

Multi-view intuitionistic fuzzy support vector machines with insensitive pinball loss for classification of noisy data

Multi-view support vector machines (MvSVMs) have been widely used to solve multi-view classification problems. However, the conventional MvSVMs often overlook the presence of noise and outliers that commonly exist in the original data. In this paper, we propose two novel multi-view intuitionistic fuzzy support vector machines with insensitive pinball loss that can not only handle the general multi-view classification problems but also be robust to noisy data. In the proposed convex optimization models, the pinball loss is incorporated into the multi-view learning, which enables the maximization of the quantization distance. Moreover, to utilize multi-view information more effectively, the intuitionistic fuzzy score is introduced to assign a weight to each multi-view sample. The intuitionistic fuzzy score combines the membership and non-membership functions, which provides an efficient mechanism to assign weights to the multi-view samples. Further, we provide a discussion of the proposed models with the state-of-the-art technologies. Experiments are conducted on a series of datasets, and the results show that the proposed convex models outperform several state-of-the-art models. The code is available at 1.

Generative Service Provisioning for IoT Devices Using Line Graph Structure

A service subgraph helps Internet-of-Things devices access resources in a dynamic Internet-of-Things device network. We propose a service subgraph generation method for Internet-of-Things device networks. Service subgraph generation aims to find more capable neighboring Internet-of-Things devices for service provisioning. We apply a line graph structure for an adequate representation of device resources. The line graph structure effectively represents the resources in the generated service subgraph. A general node classification problem constituting the generated service subgraph identifies the appropriate resource binding for service provisioning. A node in the service subgraph corresponds to a unique relationship between devices. Service provisioning is guaranteed by reinforcement learning based on the resource binding identified by node classification. The proposed line graph structure and resource binding significantly enhance the traditional intelligent resource allocation method. In addition, the proposed scheme can effectively attain service subgraphs with very low computational complexity. The proposed generative service provisioning generally has a significantly lower occupation probability than the swarm intelligence-based algorithm. The average value of the occupation probability is 0.49 with the proposed method. It is 0.12 lower than that of swarm intelligence-based algorithm.

Extracting Features from Textual Data in Class Imbalance Problems

We address class imbalance problems. These are classification problems where the target variable is binary, and one class dominates over the other. A central objective in these problems is to identify features that yield models with high precision/recall values, the standard yardsticks for assessing such models. Our features are extracted from the textual data inherent in such problems. We use n-gram frequencies as features and introduce a discrepancy score that measures the efficacy of an n-gram in highlighting the minority class. The frequency counts of n-grams with the highest discrepancy scores are used as features to construct models with the desired metrics. According to the best practices followed by the services industry, many customer support tickets will get audited and tagged as “contract-compliant” whereas some will be tagged as “over-delivered”. Based on in-field data, we use a random forest classifier and perform a randomized grid search over the model hyperparameters. The model scoring is performed using an scoring function. Our objective is to minimize the follow-up costs by optimizing the recall score while maintaining a base-level precision score. The final optimized model achieves an acceptable recall score while staying above the target precision. We validate our feature selection method by comparing our model with one constructed using frequency counts of n-grams chosen randomly. We propose extensions of our feature extraction method to general classification (binary and multi-class) and regression problems. The discrepancy score is one measure of dissimilarity of distributions and other (more general) measures that we formulate could potentially yield more effective models.

Associator dependent algebras and Koszul duality

We resolve a 10-year-old open question of Loday of describing Koszul operads that act on the algebra of octonions. In fact, we obtain the answer by solving a more general classification problem: we find all Koszul operads among those encoding associator dependent algebras.

Incremental Learning from Low-labelled Stream Data in Open-Set Video Face Recognition

Deep Learning approaches have brought solutions, with impressive performance, to general classification problems where wealthy of annotated data are provided for training. In contrast, less progress has been made in continual learning of a set of non-stationary classes, mainly when applied to unsupervised problems with streaming data.Here, we propose a novel incremental learning approach which combines a deep features encoder with an Open-Set Dynamic Ensembles of SVM, to tackle the problem of identifying individuals of interest (IoI) from streaming face data. From a simple weak classifier trained on a few video-frames, our method can use unsupervised operational data to enhance recognition. Our approach adapts to new patterns avoiding catastrophic forgetting and partially heals itself from miss-adaptation. Besides, to better comply with real world conditions, the system was designed to operate in an open-set setting. Results show a benefit of up to 15% F1-score increase respect to non-adaptive state-of-the-art methods.

Constrained Naïve Bayes with application to unbalanced data classification

The Naïve Bayes is a tractable and efficient approach for statistical classification. In general classification problems, the consequences of misclassifications may be rather different in different classes, making it crucial to control misclassification rates in the most critical and, in many realworld problems, minority cases, possibly at the expense of higher misclassification rates in less problematic classes. One traditional approach to address this problem consists of assigning misclassification costs to the different classes and applying the Bayes rule, by optimizing a loss function. However, fixing precise values for such misclassification costs may be problematic in realworld applications. In this paper we address the issue of misclassification for the Naïve Bayes classifier. Instead of requesting precise values of misclassification costs, threshold values are used for different performance measures. This is done by adding constraints to the optimization problem underlying the estimation process. Our findings show that, under a reasonable computational cost, indeed, the performance measures under consideration achieve the desired levels yielding a user-friendly constrained classification procedure.

Classifying Near-Threshold Enhancement Using Deep Neural Network

One of the main issues in hadron spectroscopy is to identify the origin of threshold or near-threshold enhancement. Prior to our study, there is no straightforward way of distinguishing even the lowest channel threshold-enhancement of the nucleon-nucleon system using only the cross-sections. The difficulty lies in the proximity of either a bound or virtual state pole to the threshold which creates an almost identical structure in the scattering region. Identifying the nature of the pole causing the enhancement falls under the general classification problem and supervised machine learning using a feed-forward neural network is known to excel in this task. In this study, we discuss the basic idea behind deep neural network and how it can be used to identify the nature of the pole causing the enhancement. The applicability of the trained network can be explored by using an exact separable potential model to generate a validation dataset. We find that within some acceptable range of the cut-off parameter, the neural network gives high accuracy of inference. The result also reveals the important role played by the background singularities in the training dataset. Finally, we apply the method to nucleon-nucleon scattering data and show that the network was able to give the correct nature of pole, i.e. virtual pole for $${}^1S_0$$ partial cross-section and bound state pole for $${}^3S_0$$ .

Mass-conserving diffusion-based dynamics on graphs

An emerging technique in image segmentation, semi-supervised learning and general classification problems concerns the use of phase-separating flows defined on finite graphs. This technique was pioneered in Bertozzi and Flenner (2012, Multiscale Modeling and Simulation10(3), 1090–1118), which used the Allen–Cahn flow on a graph, and was then extended in Merkurjev et al. (2013, SIAM J. Imaging Sci.6(4), 1903–1930) using instead the Merriman–Bence–Osher (MBO) scheme on a graph. In previous work by the authors, Budd and Van Gennip (2020, SIAM J. Math. Anal.52(5), 4101–4139), we gave a theoretical justification for this use of the MBO scheme in place of Allen–Cahn flow, showing that the MBO scheme is a special case of a ‘semi-discrete’ numerical scheme for Allen–Cahn flow. In this paper, we extend this earlier work, showing that this link via the semi-discrete scheme is robust to passing to the mass-conserving case. Inspired by Rubinstein and Sternberg (1992, IMA J. Appl. Math.48, 249–264), we define a mass-conserving Allen–Cahn equation on a graph. Then, with the help of the tools of convex optimisation, we show that our earlier machinery can be applied to derive the mass-conserving MBO scheme on a graph as a special case of a semi-discrete scheme for mass-conserving Allen–Cahn. We give a theoretical analysis of this flow and scheme, proving various desired properties like existence and uniqueness of the flow and convergence of the scheme, and also show that the semi-discrete scheme yields a choice function for solutions to the mass-conserving MBO scheme.

Machine Learning Using Rapidity-Mass Matrices for Event Classification Problems in HEP

In this work, supervised artificial neural networks (ANN) with rapidity–mass matrix (RMM) inputs are studied using several Monte Carlo event samples for various pp collision processes. The study shows the usability of this approach for general event classification problems. The proposed standardization of the ANN feature space can simplify searches for signatures of new physics at the Large Hadron Collider (LHC) when using machine learning techniques. In particular, we illustrate how to improve signal-over-background ratios in the search for new physics, how to filter out Standard Model events for model-agnostic searches, and how to separate gluon and quark jets for Standard Model measurements.

Multi-View Support Vector Machines with the Consensus and Complementarity Information

Multi-view learning (MVL) is an active direction in machine learning that aims at exploiting the consensus and complementarity information among multiple distinct feature sets to boost the generalization performance of the counterpart algorithm. So far, two classical SVM-based MVL methods are SVM-2K and multi-view twin support vector machine (MvTSVM). They are designed only for two-view classification and cannot tackle the general multi-view classification problem. They also cannot effectively leverage the complementarity information among different feature views. In this paper, we propose two novel multi-view support vector machines with the consensus and complementarity information for MVL that not only can deal with the two-view classification problem but also the general multi-view classification problem by jointly learning multiple different views in a non-pairwise way. The disagreement among different views is regarded as a constraint or a regularization term in the objective function which plays an important role in exploring the consensus information. Combination weights for the reconstruction of each view in regularization terms are learned to explore complementarity information among different views. Finally, an efficient iteration algorithm with the classical convex quadratic programming is developed for optimization. Experimental results validate the effectiveness of our proposed methods.

Performance Boosting of Scale and Rotation Invariant Human Activity Recognition (HAR) with LSTM Networks Using Low Dimensional 3D Posture Data in Egocentric Coordinates

Human activity recognition (HAR) has been an active area in computer vision with a broad range of applications, such as education, security surveillance, and healthcare. HAR is a general time series classification problem. LSTMs are widely used for time series classification tasks. However, they work well with high-dimensional feature vectors, which reduce the processing speed of LSTM in real-time applications. Therefore, dimension reduction is required to create low-dimensional feature space. As it is experimented in previous study, LSTM with dimension reduction yielded the worst performance among other classifiers, which are not deep learning methods. Therefore, in this paper, a novel scale and rotation invariant human activity recognition system, which can also work in low dimensional feature space is presented. For this purpose, Kinect depth sensor is employed to obtain skeleton joints. Since angles are used, proposed system is already scale invariant. In order to provide rotation invariance, body relative direction in egocentric coordinates is calculated. The 3D vector between right hip and left hip is used to get the horizontal axis and its cross product with the vertical axis of global coordinate system assumed to be the depth axis of the proposed local coordinate system. Instead of using 3D joint angles, 8 number of limbs and their corresponding 3D angles with X, Y, and Z axes of the proposed coordinate system are compressed with several dimension reduction methods such as averaging filter, Haar wavelet transform (HWT), and discrete cosine transform (DCT) and employed as the feature vector. Finally, extracted features are trained and tested with LSTM (long short-term memory) network, which is an artificial recurrent neural network (RNN) architecture. Experimental and benchmarking results indicate that proposed framework boosts the performance of LSTM by approximately 30% accuracy in low-dimensional feature space.

ConvXGB: A new deep learning model for classification problems based on CNN and XGBoost

We describe a new deep learning model - Convolutional eXtreme Gradient Boosting (ConvXGB) for classification problems based on convolutional neural nets and Chen et al.’s XGBoost. As well as image data, ConvXGB also supports the general classification problems, with a data preprocessing module. ConvXGB consists of several stacked convolutional layers to learn the features of the input and is able to learn features automatically, followed by XGBoost in the last layer for predicting the class labels. The ConvXGB model is simplified by reducing the number of parameters under appropriate conditions, since it is not necessary re-adjust the weight values in a back propagation cycle. Experiments on several data sets from UCL Repository, including images and general data sets, showed that our model handled the classification problems, for all the tested data sets, slightly better than CNN and XGBoost alone and was sometimes significantly better.