Local Classifiers Research Articles

Surviving in Diverse Biases: Unbiased Dataset Acquisition in Online Data Market for Fair Model Training

The online data markets have emerged as a valuable source of diverse datasets for training machine learning (ML) models. However, datasets from different data providers may exhibit varying levels of bias with respect to certain sensitive attributes in the population (such as race, sex, age, and marital status). Recent dataset acquisition research has focused on maximizing accuracy improvements for downstream model training, ignoring the negative impact of biases in the acquired datasets, which can lead to an unfair model. Can a consumer obtain an unbiased dataset from datasets with diverse biases? In this work, we propose a fairness-aware data acquisition framework (FAIRDA) to acquire high-quality datasets that maximize both accuracy and fairness for consumer local classifier training while remaining within a limited budget. Given the biases of data commodities remain opaque to consumers, the data acquisition in FAIRDA employs explore-exploit strategies. Based on whether exploration and exploitation are conducted sequentially or alternately, we introduce two algorithms: the knowledge-based offline data acquisition (KDA) and the reward-based online data acquisition algorithms (RDA). Each algorithm is tailored to specific customer needs, giving the former an advantage in computational efficiency and the latter an advantage in robustness. We conduct experiments to demonstrate the effectiveness of the proposed data acquisition framework in steering users toward fairer model training compared to existing baselines under varying market settings.

Metaheuristic-driven space partitioning and ensemble learning for imbalanced classification

Imbalanced classification is a common issue in Machine Learning, particularly when misclassifying minor instances leads to significant costs. In literature, various strategies have been employed to address this problem. These include data-level, algorithm-level, cost-sensitive, and hybrid-level algorithms designed to tackle imbalanced problems. This paper aims to introduce a novel method that simultaneously enhances the ability of classification models to identify patterns more effectively and addresses imbalanced problems while minimizing alterations to the original data distribution. Our proposed framework combines ensemble learning, space partitioning, and the Synthetic Minority Oversampling Technique (SMOTE). This method decomposes the space into balanced sub-spaces and then trains an ensemble classifier based on these sub-spaces using a bagging approach. In the initial step, we develop a Space Partitioning by Metaheuristic algorithm (SPMH) to divide the space into multiple balanced subspaces. In the subsequent step, we present Imbalanced Classification by SPMH (ICSPMH) as a solution to imbalanced class problems. ICSPMH uses SPMH multiple times to divide the space into different sub-spaces, creating various sub-spaces each time. It then trains different classifiers for each portion of the space, creating an ensemble classifier through a bagging technique. To assess the performance of our proposed framework, we selected 44 well-known datasets for comparison with some state-of-the-art approaches. The results demonstrate that ICSPMH outperforms other competent methods and can potentially reduce the oversampling rate to zero. Additionally, an experiment indicated that the choice of metaheuristic algorithm in SPMH does not significantly impact the final performance. The paper also includes a correlation analysis between oversampling rate and final performance, revealing that the framework effectively eliminates imbalanced data problems with minimal changes to the original dataset. In summary, because ICSPMH applies fewer changes in data distribution and sets up local classifiers that improve classification performance, it looks like a promising method for classifying imbalanced datasets.

Local means-based fuzzy k-nearest neighbor classifier with Minkowski distance and relevance-complementarity feature weighting

This paper introduces an enhanced fuzzy k-nearest neighbor (FKNN) approach called the feature-weighted Minkowski distance and local means-based fuzzy k-nearest neighbor (FWM-LMFKNN). This method improves classification accuracy by incorporating feature weights, Minkowski distance, and class representative local mean vectors. The feature weighting process is developed based on feature relevance and complementarity. We improve the distance calculations between instances by utilizing feature information-based weighting and Minkowski distance, resulting in a more precise set of nearest neighbors. Furthermore, the FWM-LMFKNN classifier considers the local structure of class subsets by using local mean vectors instead of individual neighbors, which improves its classification performance. Empirical results using twenty different real-world data sets demonstrate that the proposed method achieves statistically significantly higher classification performance than traditional KNN, FKNN, and six other related state-of-the-art methods.

RAAFNet: Reverse Attention Adaptive Fusion Network for Large-Scale Point Cloud Semantic Segmentation

Point cloud semantic segmentation is essential for comprehending and analyzing scenes. However, performing semantic segmentation on large-scale point clouds presents challenges, including demanding high memory requirements, a lack of structured data, and the absence of topological information. This paper presents a novel method based on the Reverse Attention Adaptive Fusion network (RAAFNet) for segmenting large-scale point clouds. RAAFNet consists of a reverse attention encoder–decoder module, an adaptive fusion module, and a local feature aggregation module. The reverse attention encoder–decoder module is applied to extract point cloud features at different scales. The adaptive fusion module enhances fine-grained representation within multi-resolution feature maps. Furthermore, a local aggregation classifier is introduced, which aggregates the features of neighboring points to the center point in order to leverage contextual information and enhance the classifier’s perceptual capability. Finally, the predicted labels are generated. Notably, our method excels at extracting point cloud features across different dimensions and produces highly accurate segmentation results. Experimental results on the Semantic3D dataset achieved an overall accuracy of 89.9% and a mIoU of 74.4%.

Identification and Discrimination of Petrol Sources by Nuclear Magnetic Resonance Spectroscopy and Machine Learning in Fire Debris Analysis

Petrol is considered the most common fire accelerant. However, the identification and classification of petrol sources through the years has proven to be a challenging field in the investigation of fire debris analysis. This research explored the possibility of identifying petrol sources by high-field NMR methods accompanied by ML (machine learning). The automated identification and classification of petrol brands were achieved for first time based on the ML classification model developed in this research. A hierarchical classification model was constructed using local classifiers to categorize neat or weathered petrol into its sources.

Top-Heavy CapsNets Based on Spatiotemporal Non-Local for Action Recognition

To effectively comprehend human actions, we have developed a Deep Neural Network (DNN) that utilizes inner spatiotemporal non-locality to capture meaningful semantic context for efficient action identification. This work introduces the Top-Heavy CapsNet as a novel approach for video analysis, incorporating a 3D Convolutional Neural Network (3DCNN) to apply the thematic actions of local classifiers for effective classification based on motion from the spatiotemporal context in videos. This DNN comprises multiple layers, including 3D Convolutional Neural Network (3DCNN), Spatial Depth-Based Non-Local (SBN) layer, and Deep Capsule (DCapsNet). Firstly, the 3DCNN extracts structured and semantic information from RGB and optical flow streams. Secondly, the SBN layer processes feature blocks with spatial depth to emphasize visually advantageous cues, potentially aiding in action differentiation. Finally, DCapsNet is more effective in exploiting vectorized prominent features to represent objects and various action features for the ultimate label determination. Experimental results demonstrate that the proposed DNN achieves an average accuracy of 97.6%, surpassing conventional DNNs on the traffic police dataset. Furthermore, the proposed DNN attains average accuracies of 98.3% and 80.7% on the UCF101 and HMDB51 datasets, respectively. This underscores the applicability of the proposed DNN for effectively recognizing diverse actions performed by subjects in videos.

Classification of complex local environments in systems of particle shapes through shape symmetry-encoded data augmentation.

Detecting and analyzing the local environment is crucial for investigating the dynamical processes of crystal nucleation and shape colloidal particle self-assembly. Recent developments in machine learning provide a promising avenue for better order parameters in complex systems that are challenging to study using traditional approaches. However, the application of machine learning to self-assembly on systems of particle shapes is still underexplored. To address this gap, we propose a simple, physics-agnostic, yet powerful approach that involves training a multilayer perceptron (MLP) as a local environment classifier for systems of particle shapes, using input features such as particle distances and orientations. Our MLP classifier is trained in a supervised manner with a shape symmetry-encoded data augmentation technique without the need for any conventional roto-translations invariant symmetry functions. We evaluate the performance of our classifiers on four different scenarios involving self-assembly of cubic structures, two-dimensional and three-dimensional patchy particle shape systems, hexagonal bipyramids with varying aspect ratios, and truncated shapes with different degrees of truncation. The proposed training process and data augmentation technique are both straightforward and flexible, enabling easy application of the classifier to other processes involving particle orientations. Our work thus presents a valuable tool for investigating self-assembly processes on systems of particle shapes, with potential applications in structure identification of any particle-based or molecular system where orientations can be defined.

Optimization techniques for IDS‐Generated traffic congestion control in VANET

AbstractVehicular Ad‐hoc Network (VANET) is an emerging field of wireless networks that enables a variety of vehicle safety and convenience applications. It employs Intrusion Detection System (IDS) frameworks in its different tiers to ensure reliable and secure communication among nodes. However, IDS requires a significant amount of data to process for monitoring intrusive activities in the network. As a result, the volume of traffic increases, resulting in the network congestion. Motivated by this fact, this study provides an overview of the optimization techniques for VANET traffic congestion control. It discusses a state‐of‐the‐art analysis along with the requirements for IDS‐generated traffic congestion control. It highlights the congestion control approaches for the traffic generated by an IDS and identifies the challenges in this domain. This study also proposes a novel IDS framework for reducing IDS‐generated network traffic by combining the Local Outlier Factor and Random Forest classifier. The proposed study achieved a high precision while yielding low false positive and false negative rates. The study outperformed the existing studies with an increase in accuracy of 1.16% and a reduction in attack detection time of 1.1869 seconds. Additionally, it discusses the possible future research directions that can be applied to address the issues of IDS‐generated traffic congestion. Overall, this study serves as a comprehensive guide to the current status of IDS‐generated traffic congestion control and diverse approaches to lessen it that can be employed by academicians and researchers.

Federated Label-Noise Learning with Local Diversity Product Regularization

Training data in federated learning (FL) frameworks can have label noise, since they must be stored and annotated on clients' devices. If trained over such corrupted data, the models learn the wrong knowledge of label noise, which highly degrades their performance. Although several FL schemes are designed to combat label noise, they suffer performance degradation when the clients' devices only have limited local training samples. To this end, a new scheme called federated label-noise learning (FedLNL) is developed in this paper. The key problem of FedLNL is how to estimate a noise transition matrix (NTM) accurately in the case of limited local training samples. If a gradient-based update method is used to update the local NTM on each client's device, it can generate too large gradients for the local NTM, causing a high estimation error of the local NTM. To tackle this issue, an alternating update method for the local NTM and the local classifier is designed in FedLNL, where the local NTM is updated by a Bayesian inference-based update method. Such an alternating update method makes the loss function of existing NTM-based schemes not applicable to FedLNL. To enable federated optimization of FedLNL, a new regularizer on the parameters of the classifier called local diversity product regularizer is designed for the loss function of FedLNL. The results show that FedLNL improves the test accuracy of a trained model by up to 25.98%, compared with the state-of-the-art FL schemes that tackle label-noise issues.

Local dual-graph discriminant classifier for binary classification

Graph-based methods mine the potential structural information of data by constructing various graphs that positively affect the classifiers when dealing with classification problems. However, traditional graph-based classifiers are the most common single-graph classifiers and minimize only intra-class compactness, where inter-class separability is replaced by other factors. To consider real inter-class separability, we introduce a novel local dual-graph structure that can fully mine the geometric distribution of data by simultaneously maximizing the inter-class separability and minimizing the intra-class compactness. This local dual-graph structure reflects the relationship between samples and their neighbors and hence avoids the negative impact of outliers on the construction of graphs. Furthermore, a novel classifier called the local dual-graph discriminant classifier (LDGDC) is proposed using a local dual-graph structure. Originally, LDGDC is designed to perform the following optimization: minimization of the 2-norm regularization of model coefficients and intra-class compactness, and maximization of the inter-class separability, which is a non-convex optimization problem. To facilitate the solution, we transform the original non-convex problem of LDGDC into a convex problem. Finally, experiments were conducted on several public datasets, and the results demonstrate the effectiveness and robustness of the proposed LDGDC.

Semantic Segmentation of 3D Point Clouds in Outdoor Environments Based on Local Dual-Enhancement

Semantic segmentation of 3D point clouds in drivable areas is very important for unmanned vehicles. Due to the imbalance between the size of various outdoor scene objects and the sample size, the object boundaries are not clear, and small sample features cannot be extracted. As a result, the semantic segmentation accuracy of 3D point clouds in outdoor environment is not high. To solve these problems, we propose a local dual-enhancement network (LDE-Net) for semantic segmentation of 3D point clouds in outdoor environments for unmanned vehicles. The network is composed of local-global feature extraction modules, and a local feature aggregation classifier. The local-global feature extraction module captures both local and global features, which can improve the accuracy and robustness of semantic segmentation. The local feature aggregation classifier considers the feature information of neighboring points to ensure clarity of object boundaries and the high overall accuracy of semantic segmentation. Experimental results show that provides clearer boundaries between various objects, and has higher identification accuracy for small sample objects. The LDE-Net has good performance for semantic segmentation of 3D point clouds in outdoor environments.

Development of a Fault Detection and Localization Model for a Water Distribution Network

Water distribution networks are complex systems that aid in the delivery of water to residential and non-residential areas. However, the networks can be affected by different types of faults, which could lead to the wastage of treated water. As such, there is a need to develop a reliable leakage detection and localization system that can detect leak occurrences in the network. This study, using a simulated dataset from EPANET, presents the application of supervised machine learning classifiers for leak detection and localization in the water distribution network of the University of Port Harcourt Choba campus. The study compared three machine learning classification tools that are used in pattern recognition analysis: the support vector machine, k-nearest neighbor, and artificial neural network. The robustness and effectiveness of the proposed approach are compared with those of the performance of the classifiers for leakage detection in the network of the case study. The results show that the support vector machine performs the best, with 79% accuracy, while the respective accuracies for the remaining classifiers are 70% for the k-nearest neighbor and 61% for the artificial neural networks. The high accuracy demonstrated by the models shows that they are able to detect and address issues relating to fault detection in a water distribution network. This model could provide a leakage detection system to be applied to buildings for the efficient management of water in their networks.

Oral_voting_transfer: classification of oral microorganisms' function proteins with voting transfer model.

The oral microbial group typically represents the human body's highly complex microbial group ecosystem. Oral microorganisms take part in human diseases, including Oral cavity inflammation, mucosal disease, periodontal disease, tooth decay, and oral cancer. On the other hand, oral microbes can also cause endocrine disorders, digestive function, and nerve function disorders, such as diabetes, digestive system diseases, and Alzheimer's disease. It was noted that the proteins of oral microbes play significant roles in these serious diseases. Having a good knowledge of oral microbes can be helpful in analyzing the procession of related diseases. Moreover, the high-dimensional features and imbalanced data lead to the complexity of oral microbial issues, which can hardly be solved with traditional experimental methods. To deal with these challenges, we proposed a novel method, which is oral_voting_transfer, to deal with such classification issues in the field of oral microorganisms. Such a method employed three features to classify the five oral microorganisms, including Streptococcus mutans, Staphylococcus aureus, abiotrophy adjacent, bifidobacterial, and Capnocytophaga. Firstly, we utilized the highly effective model, which successfully classifies the organelle's proteins and transfers to deal with the oral microorganisms. And then, some classification methods can be treated as the local classifiers in this work. Finally, the results are voting from the transfer classifiers and the voting ones. The proposed method achieved the well performances in the five oral microorganisms. The oral_voting_transfer is a standalone tool, and all its source codes are publicly available at https://github.com/baowz12345/voting_transfer.

Development of a 30 m resolution global sand dune/sheet classification map (GSDS30) using multi-source remote sensing data

Accurate information of sand dune/sheet (SDS) spatial distribution is required by global-scale environmental assessment. However, the great spatio-temporal heterogeneity of SDS and the confusion of SDS to similar land cover types lead to variant feature representations and inadequate classification samples; thus, existing SDS mapping focuses on the regional scale, and the high-quality global SDS map is still lacking. In this study, we proposed a classification strategy and for the first time developed a 30 m resolution global SDS map (GSDS30) in the year 2017 that contained two SDS types, i.e., shifting SDS and semi-fixed/fixed SDS. The proposed strategy started by determining the initial mapping extent of SDS. Then, multi-source features were extracted to capture SDS variations and enlarge the separability of diverse land covers. Thirdly, we developed a prior-constraining K-nearest neighbor method to collect global SDS samples. Finally, a local random forest classifier was applied to generate classification results. The evaluation based on two validation sample sets showed good performance on the overall classification accuracy (88.73% and 87.92%), the Kappa coefficient (0.852 and 0.843), and the producer's and user's accuracies for two SDS types. Based on GSDS30, we found that global SDS occupied an area of 10.43 million km2, in which shifting SDS accounted for 60.41% and semi-fixed/fixed SDS accounted for the remaining 39.59%. The majority of SDS was distributed in Africa, Asia, and Oceania, where Australia had the greatest area of SDS. The combination of GSDS30 and population data showed that global SDS was inhabited by 14.5 million people, and 94.95% of them lived in semi-fixed/fixed SDS. This study is the first attempt to automatically produce global SDS map, which is not only necessary for representing the distribution of SDS, but also valuable in managing aeolian desertification and promoting sustainable development.

Effective Feature Extraction Using Residual Attention and Local Context Aware Classifier for Crop Yield Prediction

Effective Feature Extraction Using Residual Attention and Local Context Aware Classifier for Crop Yield Prediction

Vision-Language Navigation With Beam-Constrained Global Normalization.

Vision-language navigation (VLN) is a challenging task, which guides an agent to navigate in a realistic environment by natural language instructions. Sequence-to-sequence modeling is one of the most prospective architectures for the task, which achieves the agent navigation goal by a sequence of moving actions. The line of work has led to the state-of-the-art performance. Recently, several studies showed that the beam-search decoding during the inference can result in promising performance, as it ranks multiple candidate trajectories by scoring each trajectory as a whole. However, the trajectory-level score might be seriously biased during ranking. The score is a simple averaging of individual unit scores of the target-sequence actions, and these unit scores could be incomparable among different trajectories since they are calculated by a local discriminant classifier. To address this problem, we propose a global normalization strategy to rescale the scores at the trajectory level. Concretely, we present two global score functions to rerank all candidates in the output beam, resulting in more comparable trajectory scores. In this way, the bias problem can be greatly alleviated. We conduct experiments on the benchmark room-to-room (R2R) dataset of VLN to verify our method, and the results show that the proposed global method is effective, providing significant performance than the corresponding baselines. Our final model can achieve competitive performance on the VLN leaderboard.

Federated Discriminative Representation Learning for Image Classification.

Acquiring big-size datasets to raise the performance of deep models has become one of the most critical problems in representation learning (RL) techniques, which is the core potential of the emerging paradigm of federated learning (FL). However, most current FL models concentrate on seeking an identical model for isolated clients and thus fail to make full use of the data specificity between clients. To enhance the classification performance of each client, this study introduces the FDRL, a federated discriminative RL model, by partitioning the data features of each client into a global subspace and a local subspace. More specifically, FDRL learns the global representation for federated communication between those isolated clients, which is to capture common features from all protected datasets via model sharing, and local representations for personalization in each client, which is to preserve specific features of clients via model differentiating. Toward this goal, FDRL in each client trains a shared submodel for federated communication and, meanwhile, a not-shared submodel for locality preservation, in which the two models partition client-feature space by maximizing their differences, followed by a linear model fed with combined features for image classification. The proposed model is implemented with neural networks and optimized in an iterative manner between the server of computing the global model and the clients of learning the local classifiers. Thanks to the powerful capability of local feature preservation, FDRL leads to more discriminative data representations than the compared FL models. Experimental results on public datasets demonstrate that our FDRL benefits from the subspace partition and achieves better performance on federated image classification than the state-of-the-art FL models.

БАГАТОРІВНЕВА СИСТЕМА РОЗПІЗНАВАННЯ ОБЛИЧЧЯ

The problem of biometric person identification based on component-based face recognition is considered. It is shown that the face recognition system can be represented as a hierarchically organized multilevel system in which an ensemble of local classifiers forms “soft” decisions about the belonging of images of individual components of the face to given classes. Then, based on the integration of these decisions, the formation of the final decision on whether the recognized face belongs to one of the given classes is realized. The problems of constructing a model of a local classifier, as well as choosing an integrator of intermediate solutions of local classifiers, are formulated and solved. Keywords: pattern recognition, multilevel recognition system, classifier ensemble, classifier combination rule, decision making.

ImageNet classification with Raspberry Pis: federated learning algorithms of local classifiers

PurposeThis study aims to propose novel edge device-tailored federated learning algorithms of local classifiers (stochastic gradient descent, support vector machines), namely, FL-lSGD and FL-lSVM. These algorithms are designed to address the challenge of large-scale ImageNet classification.Design/methodology/approachThe authors’ FL-lSGD and FL-lSVM trains in a parallel and incremental manner to build an ensemble local classifier on Raspberry Pis without requiring data exchange. The algorithms load small data blocks of the local training subset stored on the Raspberry Pi sequentially to train the local classifiers. The data block is split into k partitions using the k-means algorithm, and models are trained in parallel on each data partition to enable local data classification.FindingsEmpirical test results on the ImageNet data set show that the authors’ FL-lSGD and FL-lSVM algorithms with 4 Raspberry Pis (Quad core Cortex-A72, ARM v8, 64-bit SoC @ 1.5GHz, 4GB RAM) are faster than the state-of-the-art LIBLINEAR algorithm run on a PC (Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores, 32GB RAM).Originality/valueEfficiently addressing the challenge of large-scale ImageNet classification, the authors’ novel federated learning algorithms of local classifiers have been tailored to work on the Raspberry Pi. These algorithms can handle 1,281,167 images and 1,000 classes effectively.

Explainable Artificial Intelligence Using Expressive Boolean Formulas

We propose and implement an interpretable machine learning classification model for Explainable AI (XAI) based on expressive Boolean formulas. Potential applications include credit scoring and diagnosis of medical conditions. The Boolean formula defines a rule with tunable complexity (or interpretability) according to which input data are classified. Such a formula can include any operator that can be applied to one or more Boolean variables, thus providing higher expressivity compared to more rigid rule- and tree-based approaches. The classifier is trained using native local optimization techniques, efficiently searching the space of feasible formulas. Shallow rules can be determined by fast Integer Linear Programming (ILP) or Quadratic Unconstrained Binary Optimization (QUBO) solvers, potentially powered by special-purpose hardware or quantum devices. We combine the expressivity and efficiency of the native local optimizer with the fast operation of these devices by executing non-local moves that optimize over the subtrees of the full Boolean formula. We provide extensive numerical benchmarking results featuring several baselines on well-known public datasets. Based on the results, we find that the native local rule classifier is generally competitive with the other classifiers. The addition of non-local moves achieves similar results with fewer iterations. Therefore, using specialized or quantum hardware could lead to a significant speedup through the rapid proposal of non-local moves.