Optimum-Path Forest Classifier Research Articles

A comprehensive study among distance measures on supervised optimum-path forest classification

Supervised pattern classification relies on a labeled training set to learn decision boundaries that separate samples from different classes. Such samples can be either weakly- or reliably-labeled; in the first case, one can employ techniques specifically designed to cope with uncertainty during labeling, and in the other scenario, it relies on numerous alternatives, including metric learning. Pattern classifiers usually adopt the Euclidean distance to compare samples and assess their proximity, but this implies the feature space is embedded in a plane. However, samples are embedded in curved spaces for some applications, although not straightforward to prove. In this manuscript, we assessed the performance of the Optimum-Path Forest (OPF) classifier under different distance functions, which are used to weigh arcs among samples, for a graph encoding the feature space. This work compared 47 distance measures applied to the OPF classifier considering 22 datasets, plus Decision Trees, Logistic Regression, and Support Vector Machines. The experiments highlighted that OPF is user-friendly when handling distance measures and can obtain better accuracies in some situations than its standard (Euclidean) counterpart and the classifiers mentioned above. On the other hand, time-consuming distance calculations may affect OPF’s efficiency during inference.

Weakly supervised classification through manifold learning and rank-based contextual measures

Over the last decade, significant advances have been achieved by machine learning approaches, notably in supervised learning scenarios. Supported by the advent of deep learning and comprehensive training sets, the accuracy achieved on classification tasks has improved significantly. Simultaneously, we have experienced massive growth in multimedia data and applications, which have become ubiquitous in several domains. However, with the increase in multimedia data collections, significant bottlenecks associated with the lack of labeled data emerged. To surpass this critical issue, developing methods capable of exploiting the unlabeled data and operating under weak supervision has become imperative. This work proposes a rank-based model capable of using contextual information encoded in the unlabeled data to perform weakly supervised classification. We evaluated the proposed weakly supervised approach on multimedia classification tasks with and without manifold learning algorithms, considering several combinations of rank correlation measures and classifiers. An experimental evaluation was conducted on 6 public image datasets considering different features, including convolutional neural networks and visual transformers. Positive gains were achieved compared to supervised and semi-supervised baselines for the same amount of labeled data. For instance, the proposed approach with manifold learning enhanced the accuracy of the Optimum-Path Forest (OPF) classifier from 71.77% to 83.24% when applied to the Flowers dataset and ResNet features. Among the conclusions, this work reveals that rank-based correlation measures and manifold learning can be used for a more effective labeled set expansion.

Measuring the quality of projections of high-dimensional labeled data

Dimensionality reduction techniques, also called projections, are one of the main tools for visualizing high-dimensional data. To compare such techniques, several quality metrics have been proposed. However, such metrics may not capture the visual separation among groups/classes of samples in a projection, i.e., having groups of similar (same label) points far from other (distinct label) groups of points. For this, we propose a pseudo-labeling mechanism to assess visual separation using the performance of a semi-supervised optimum-path forest classifier (OPFSemi), measured by Cohen’s Kappa. We argue that lower label propagation errors by OPFSemi in projections are related to higher data/visual separation. OPFSemi explores local and global information of data distribution when computing optimum connectivity between samples in a projection for label propagation. It is parameter-free, fast to compute, easy to implement, and generically handles any high-dimensional quantitative labeled dataset and projection technique. We compare our approach with four commonly used scalar metrics in the literature for 18 datasets and 39 projection techniques. Our results consistently show that our proposed metric consistently scores values in line with the perceived visual separation, surpassing existing projection-quality metrics in this respect.

Deep feature annotation by iterative meta-pseudo-labeling on 2D projections

The absence of large annotated datasets to train deep neural networks (DNNs) is an issue since manual annotation is time-consuming, expensive, and error-prone. Semi-supervised learning techniques can address the problem propagating pseudo labels from supervised to unsupervised samples. However, they still require training and validation sets with many supervised samples. This work proposes a methodology, namely Deep Feature Annotation (DeepFA), that dismisses the validation set and uses very few supervised samples (e.g., 1% of the dataset). DeepFA modifies the feature spaces of a DNN along with meta-pseudo-labeling iterations in a 2D non-linear projection space using the most confidently labeled samples of an optimum-path forest semi-supervised classifier. We present a comprehensive study on DeepFA and a new variant that detects the best DNN model for generalization during the pseudo-labeling iterations. We evaluate components of DeepFA on eight datasets, finding the best DeepFA approach and showing that it outperforms self-pseudo-labeling.

Feature selection algorithm based on binary BAT algorithm and optimum path forest classifier for breast cancer detection using both echographic and elastographic mode ultrasound images.

Breast cancer is one of the fatal diseases among women. Every year, its incidence and mortality rate increase globally. Mammography and sonography are widely used in breast cancer detection. Because mammography misses many cancers and shows false negatives in the denser tissues, sonography is preferred to give some extra information in addition to that available from mammography. To improve the performance of breast cancer detection by reducing false positives. The local binary pattern (LBP) texture features must be extracted from ultrasound elastographic and echographic images of the same patients and then fused to form a single feature vector. Local Binary Pattern (LBP) texture features of elastographic and echographic images are extracted, and reduced individually through a hybrid feature selection technique based on binary BAT algorithm (BBA) and optimum path forest (OPF) classifier and then fused serially. Finally, the support vector machine classifier is used to classify the final fused feature set. Various relevant performance metrics such as accuracy, sensitivity, specificity, discriminant power, Mathews correlation coefficient (MCC), F1 score, and Kappa were used to analyze the classification results. The use of LBP feature produces 93.2% accuracy, 94.4% sensitivity, 92.3% specificity, 89.5% precision value, 91.88% F1 score, 93.34% balanced classification rate, and Mathews correlation coefficient of 0.861. The performance was compared with gray level co-occurrence matrix (GLCM), gray level difference matrix (GLDM), and LAWs features, and showed that LBP outperformed. Because the specificity is better, this method could be useful for detecting breast cancer with minimum false negatives.

Intelligent IoT security monitoring based on fuzzy optimum-path forest classifier

Detection of intrusions in Internet of Things networks is essential to maintain the availability and integrity of the data generated and transmitted by connected devices. Such a procedure is paramount when the data originate from critical activities, such as military, financial, industrial, and health sectors. In the last decades, machine learning (ML)-based approaches have become one of the most suitable and adopted procedures for the task, providing automatic, fast, and accurate results. Despite such success, the literature still presents a gap regarding valid applications of intrusion detection in the IoT environments, which usually stands for a challenging task composed of different types of attacks. In this context, this work applies a recent technique based on graphs and logic fuzzy, namely Fuzzy Optimum-Path Forest (Fuzzy OPF), to detect threats that escape an IoT network’s regular traffic. We evaluate our model against five well-known ML algorithms, i.e., Linear Discriminant Analysis, Support Vector Machine, Naive Bayes, K-Nearest Neighbors, and the standard Optimum-Path Forest. Experimental results show that Fuzzy OPF outperforms the baselines considering accuracy, recall, and F1 metrics. As a result, the Fuzzy OPF proposal for intrusion detection had a hit rate of 98 and 99%.

Improved breast cancer detection using fusion of bimodal sonographic features through binary firefly algorithm

ABSTRACT Breast cancer is a major health issue and causes a larger number of deaths in women. As per GLOBOCAN statistics, breast cancer has surpassed lung cancer in 2020 with 2.3 million anticipated new cases which is 11.7% of total cancer cases. Sonography is used in addition to mammography to detect, describe and locate breast lesions effectively. Elastography and Echography are widely used sonographic techniques as they provide different and distinctive information for identifying breast diseases. To obtain improved detection performance, this work focuses on the fusion of Local Binary Pattern (LBP) texture features from ultrasound elastogram and echogram images followed by feature selection using Binary Firefly Algorithm (BFA) with Optimum Path Forest (OPF) classifier accuracy as a fitness function for feature selection. This method produces 97.3% accuracy, 96.2% sensitivity, 98.2% specificity, 97.3% precision, 96.2% F1 score, 94.71% Balanced Classification Rate, and Mathews Correlation Coefficient of 0.884 outperforming existing works.

A Comprehensive Study Among Distance Measures on Supervised Optimum-Path Forest Classification

A Comprehensive Study Among Distance Measures on Supervised Optimum-Path Forest Classification

An incremental Optimum-Path Forest classifier and its application to non-technical losses identification

Non-technical losses stand for the energy consumed but not billed, affecting the energy grid as a whole. Such an issue somehow prevails in developing countries, harming the quality of energy and preventing social programs benefit from tax revenues. Machine learning techniques can help mitigate it by mining information from fraudsters and legal users for further decision-making. In this paper, we deal with a steady increase of dataset size, i.e., the incremental learning problem, which can cope with datasets regularly provided by energy companies, requiring the learner to be updated constantly. Since repeating the entire learning process might be prohibitive, adjusting the model to the new data shows to be a better choice. We propose an incremental Optimum-Path Forest approach with k-nn neighborhood that is considerably more efficient for training than its counterpart version, with experiments validated in general-purpose datasets and also in the context of non-technical losses identification.

Optimum-path forest stacking-based ensemble for intrusion detection

Machine learning techniques have been extensively researched in the last years, mainly due to their effectiveness when dealing with recognition or classification applications. Typically, one can comprehend using a Machine Learning system to autonomously delegate routines, save human efforts, and produce great insights regarding decision-making tasks. This paper introduces and validates a stacking-based ensemble approach using Optimum-Path Forest classifiers in intrusion detection tasks. Instead of only using the famous NSL-KDD dataset, we propose a new dataset called uneSPY, which we believe will fill the gap concerning new intrusion detection datasets. Both datasets were evaluated under several classifiers, including Logistic Regression, Decision Trees, Support Vector Machines, Optimum-Path Forests, and compared against Optimum-Path Forest stacking-based ensembles. Experimental results showed an Optimum-Path Forest stacking-based ensemble classification suitability, particularly when considering its ability to generalize large volumes of data while sustaining its performance.

Hierarchical learning using deep optimum-path forest

Bag-of-Visual Words (BoVW) and deep learning techniques have been widely used in several domains, which include computer-assisted medical diagnoses. In this work, we are interested in developing tools for the automatic identification of Parkinson’s disease using machine learning and the concept of BoVW. The proposed approach concerns a hierarchical-based learning technique to design visual dictionaries through the Deep Optimum-Path Forest classifier. The proposed method was evaluated in six datasets derived from data collected from individuals when performing handwriting exams. Experimental results showed the potential of the technique, with robust achievements.

Remote Sensing Applied to the Extraction of Road Geometric Features Based on Optimum Path Forest Classifiers, Northeastern Brazil

One of the principal difficulties related to road safety management in Brazil is the lack of data on road projects, especially those on rural roads, which makes it difficult to use road safety studies and models from other countries as a reference. Updating road networks through the use of hyperspectral remote sensing images can be a good alternative. However, accurately recognizing and extracting hyperspectral images from roads has been recognized as a challenging task in the processing of hyperspectral data. In order to solve the aforementioned challenges, Hyperion hyperspectral images were combined with the Optimum Forest Path (OPF) algorithm for supervised classification of rural roads and the effectiveness of the OPF and SVM classifiers when applied to these areas was compared. Both classifiers produced reasonable results, however, the OPF algorithm outperformed SVM. The higher classification accuracy obtained by the OPF was mainly attributed to the ability to better distinguish between regions of exposed soil and unpaved roads.

An Efficient Binary Clonal Selection Algorithm with Optimum Path Forest for Feature Selection

Feature selection is an important step in different applications such as data mining, classification, pattern recognition, and optimization. Until now, finding the most informative set of features among a large dataset is still an open problem. In computer science, a lot of metaphors are imported from nature and biology and proved to be efficient when applying them in an artificial way to solve a lot of problems. Examples include Neural Networks, Human Genetics, Flower Pollination, and Human Immune system. Clonal selection is one of the processes that happens in the human immune system while recognizing new infections. Mimicking this process in an artificial way resulted in a powerful algorithm, which is the Clonal Selection Algorithm. In this paper, we tried to explore the power of the Clonal Selection Algorithm in its binary form for solving the feature selection problem, we used the accuracy of the Optimum-Path Forest classifier, which is much faster than other classifiers, as a fitness function to be optimized. Experiments on three public benchmark datasets are conducted to compare the proposed Binary Clonal Selection Algorithm in conjunction with the Optimum Path Forest classifier with other four powerful algorithms. The four algorithms are Binary Flower Pollination Algorithm, Binary Bat Algorithm, Binary Cuckoo Search, and Binary Differential Evolution Algorithm. In terms of classification accuracy, experiments revealed that the proposed method outperformed the other four algorithms and moreover with a smaller number of features. Also, the proposed method took less average execution time in comparison with the other algorithms, except for Binary Cuckoo Search. The statistical analysis showed that our proposal has a significant difference in accuracy compared with the Binary Bat Algorithm and the Binary Differential Evolution Algorithm.

A Novel Approach for Optimum-Path Forest Classification Using Fuzzy Logic

In the past decades, fuzzy logic has played an essential role in many research areas. Alongside, graph-based pattern recognition has shown to be of great importance due to its flexibility in partitioning the feature space using the background from graph theory. Some years ago, a new framework for both supervised, semi-supervised, and unsupervised learning named Optimum-Path Forest (OPF) was proposed with competitive results in several applications, besides comprising a low computational burden. In this paper, we propose the Fuzzy Optimum-Path Forest, an improved version of the standard OPF classifier that learns the samples' membership in an unsupervised fashion, which are further incorporated during supervised training. Such information is used to identify the most relevant training samples, thus improving the classification step. Experiments conducted over twelve public datasets highlight the robustness of the proposed approach, which behaves similarly to standard OPF in worst-case scenarios.

Semi-supervised learning with connectivity-driven convolutional neural networks

The annotation of large datasets is an issue whose challenge increases as the number of labeled samples available to train the classifier reduces in comparison to the amount of unlabeled data. In this context, semi-supervised learning methods aim at discovering and propagating labels to unlabeled samples, such that their correct labeling can improve the classification performance. In this work, we propose a semi-supervised methodology that explores the optimum connectivity among unlabeled samples through the Optimum-Path Forest (OPF) classifier to improve the learning process of Convolution Neural Networks (CNNs). Our proposal makes use of the OPF to classify an unlabeled training set that is used to pre-train a CNN for further fine-tuning using the limited labeled data only. The proposed approach is experimentally validated on traditional datasets and provides competitive results in comparison to state-of-the-art semi-supervised learning methods.

An efficient parallel implementation for training supervised optimum-path forest classifiers

In this work, we propose and analyze parallel training algorithms for the Optimum-Path Forest (OPF) classifier. We start with a naïve parallelization approach where, following traditional sequential training that considers the supervised OPF, a priority queue is used to store the best samples at each learning iteration. The proposed approach replaces the priority queue with an array and a linear search aiming at using a parallel-friendly data structure. We show that this approach leads to less competition among threads, thus yielding a more temporal and spatial locality. Additionally, we show how the use of vectorization in distance calculations affects the overall speedup and also provide directions on the situations one can benefit from that. The experiments are carried out on five public datasets with a different number of samples and features on architectures with distinct levels of parallelism. On average, the proposed approach provides speedups of up to 11.8 × and 26 × in a 24-core Intel and 64-core AMD processors, respectively.

Particle swarm optimization based fusion of ultrasound echographic and elastographic texture features for improved breast cancer detection.

Breast cancer remains the main cause of cancer deaths among women in the world. As per the statistics, it is the most common killer disease of the new era. Since 2008, breast cancer incidences have increased by more than 20%, while mortality has increased by 14%.The statistics of breast cancer incidences as per GLOBOCAN project for the years 2008 and 2012 show an increase from 22.2 to 27% globally. In India, breast cancer accounts for25% to 31%of all cancers in women. Mammography and Sonography are the two common imaging techniques used for the diagnosis and detection of breast cancer. Since Mammography fails to spot many cancers in the dense breast tissue of young patients, Sonography is preferred as an adjunct to Mammography to identify, characterize and localize breast lesions. This work aims to improve the performance of breast cancer detection by fusing the texture features from ultrasound elastographic and echographic images through Particle Swarm Optimization. The mean classification accuracy of Optimum Path Forest Classifier is used as an objective function in PSO. Seven performance metrics were computed to study the performance of the proposed technique using GLCM, GLDM, LAWs and LBP texture features through Support Vector Machine classifier. LBP feature provides accuracy, sensitivity, specificity, precision, F1 score, Mathews Correlation Coefficient and Balanced Classification Rate as 96.2%, 94.4%, 97.4%, 96.2%, 95.29%, 0.921, 95.88% respectively. The obtained performance using LBP feature is better compared to the other three features. An improvement of 6.18% in accuracy and 11.19% in specificity were achieved when compared to those obtained with previous works.

A Probabilistic Optimum-Path Forest Classifier for Non-Technical Losses Detection

Probabilistic-driven classification techniques extend the role of traditional approaches that output labels (usually integer numbers) only. Such techniques are more fruitful when dealing with problems where one is not interested in recognition/identification only, but also into monitoring the behavior of consumers and/or machines, for instance. Therefore, by means of probability estimates, one can take decisions to work better in a number of scenarios. In this paper, we propose a probabilistic-based optimum-path forest (OPF) classifier to handle the problem of non-technical losses (NTL) detection in power distribution systems. The proposed approach is compared against naive OPF, probabilistic support vector machines, and logistic regression, showing promising results for both NTL identification and in the context of general-purpose applications.

Classification of EEG signals to detect alcoholism using machine learning techniques

The diagnosis of alcoholism is of great importance not only due to its effects on the individual and society but also the costs to the national health systems. Moreover, there are a large number of people suffering from this disease worldwide. Alcoholism has critical pathological effects on the liver, immune system, brain, and heart. Machine learning techniques are already well known for the classification of biosignals as they offer an efficient way to assist professionals in the automated diagnosis of various diseases, with high accuracy rates. This work presents the classification of alcoholic electroencephalographic (EEG) signals using Wavelet Packet Decomposition (WPD) and machine learning techniques. The experiments were realized using the minimum value, maximum value, mean, standard deviation, power value, the ratio of absolute mean and the absolute mean as features to feed the classifiers. These features were combined with the objective of exploring the feasibility of such features to classify alcoholism. The classification task was performed using Support Vector Machine (SVM), Optimum-Path Forest (OPF), Nave Bayes, k-Nearest Neighbors (k-NN) and Multi-layer Perceptron (MLP). The results showed maximum values of 99.87% for specificity, sensitivity, positive predictive value (PPV), and accuracy. These results were generated using the Nave Bayes classifier and the Biorthogonal wavelet family. A comparison with other techniques was performed aiming to validate our approach. The promising results, the inclusion of OPF classifier, and the specific combinations involving the chosen classifiers and wavelet families are the main contributions of this work. Finally, our strategy proved to be very effective in classifying alcoholic EEG signals.

Learning concept drift with ensembles of optimum-path forest-based classifiers

Concept drift methods learn patterns in non-stationary environments. Although such behavior is usually not expected in traditional classification problems, in real-world scenarios one can face them very much easier. In such a context, classifiers can be fooled and their effectiveness affected as well. Some examples include theft detection in energy distribution systems, where the consumer’s behavior may change suddenly or smoothly, or even churn prediction in mobile companies. In this paper, we introduce the Optimum-Path Forest (OPF) classifier in the context of concept drift, using decisions for concept drift handling based on a committee of OPF classifiers. We consider three distinct perspectives (three rounds of experiments with variations of streaming managements) over publics datasets, being the results compared to the ones obtained by standard OPF. We consider OPF ensemble suitable to work under these dynamic scenarios since its recognition rates were considerably better when compared to traditional OPF.