Distance-based Classification Research Articles

Improving the Performance of Open-Set Recognition with Generated Fake Data

Open-set recognition models, in addition to generalizing to unseen instances of known categories, have to identify samples of unknown classes during the training phase. The main reason the latter is much more complicated is that there is very little or no information about the properties of these unknown classes. There are methodologies available to handle the unknowns. One possible method is to construct models for them by using generated inputs labeled as unknown. Generative adversarial networks are frequently deployed to generate synthetic samples representing unknown classes to create better models for known classes. In this paper, we introduce a novel approach to improve the accuracy of recognition methods while reducing the time complexity. Instead of generating synthetic input data to train neural networks, feature vectors are generated using the output of a hidden layer. This approach results in a less complex structure for the neural network representation of the classes. A distance-based classifier implemented by a convolutional neural network is used in our implementation. Our solution’s open-set detection performance reaches an AUC value of 0.839 on the CIFAR-10 dataset, while the closed-set accuracy is 91.4%, the highest among the open-set recognition methods. The generator and discriminator networks are much smaller when generating synthetic inner features. There is no need to run these samples through the first part of the classifier with the convolutional layers. Hence, this solution not only gives better performance than generating samples in the input space but also makes it less expensive in terms of computational complexity.

Low Complexity Classification Approach for Faster-Than-Nyquist (FTN) Signaling Detection

In this letter, we investigate the use of machine learning (ML) to reduce the detection complexity of faster-than-Nyquist (FTN) signaling. In particular, we view the FTN signaling detection problem as a classification task, where the received signal is considered as an unlabeled class sample that belongs to the set of all possible classes samples. We observe that by jointly considering <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{p}$ </tex-math></inline-formula> samples, where <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{p} \ll N$ </tex-math></inline-formula> and <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N$ </tex-math></inline-formula> is the transmission block length, for the FTN signaling detection, the distance between the classes samples of any distance-based classifier increases, and hence, the detection performance improves. That said, we propose a low-complexity classifier (LCC) that exploits the ISI structure of FTN signaling to perform the classification task in <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{p}$ </tex-math></inline-formula> -dimension space. The proposed LCC consists of two stages: 1) offline pre-classification that constructs the labeled classes samples in the <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$N_{p}$ </tex-math></inline-formula> -dimensional space and 2) online classification where the detection of the received samples occurs. The proposed LCC is extended to produce soft-outputs as well. Simulation results show the effectiveness of the proposed LCC in balancing performance and complexity.

A Novel Method of Production Line Bearing Fault Diagnosis Based on 2D Image and Cross-Domain Few-Shot Learning

Data-driven intelligent fault diagnosis has made considerable strides. However, collecting sufficient fault information in real production data is extremely challenging. Therefore, a novel method of bearing fault diagnosis based on two-dimensional (2D) images and cross-domain few-shot learning is proposed. Initially, the approach uses multiscale morphology to convert the bearing’s one-dimensional (1D) vibration signal into a 2D image, which preserves the whole information. Second, to address the issue of limited bearing fault data, we extend a substantial amount of natural image knowledge to the converted 2D image based on the improved cross-domain few-shot learning method. A distance-based classifier is employed to prevent the problem of overfitting owing to insufficient data to improve the approach’s classification capacity with few samples. The experimental results demonstrate that, with the limited dataset provided, our method outperforms other prevalent methods and has high feasibility and certain engineering applications.

Performa Klasifikasi Berbasis Jarak untuk Deteksi Covid-19 Varian Delta dan Omicron Menggunakan Citra CT-Scan Paru-Paru

During the Covid-19 pandemic, there were two popular Covid-19 variants, namely Delta and Omicron. A non-laboratory approach is needed to detect the Delta and Omicron variants of Covid-19 to prevent a high risk of exposure to these two variants. This study proposes the detection of COVID-19 variants of Delta and Omicron using computerized tomography scan (CT-scan) images of the lungs using distance-based classification. There are 5 distance-based classification methods used to determine the best performance for the Delta and Omicron variant Covid-19 classification. Performance is measured based on the comparison of accuracy, precision and recall of each distance method. The distance method used in this study is Euclidean, Manhattan, Minkowski, Chebyshev, and Canberra. The dataset used was downloaded from the Kaggle database. There are 440 total CT-scan images of the lungs which are divided into 220 Covid-19 images of the Delta and Omicron variants and 220 non-Covid-19 images as training data. Meanwhile, there are test data of 140 Covid-19 images for the Delta and Omicron variants and 140 non-Covid-19 images. Based on the comparison of the performance of distance-based classification, it is concluded that the Manhattan Distance has the best performance compared to the other 4 distance methods. Manhattan distance has an accuracy of 58.57%, precision of 56.52%, and recall with a value of 74.28%. Meanwhile, the lowest accuracy value is owned by the Eucliean Distance of 48.21%. Then, the Minkowski distance has the lowest precision and recall with values ​​of 48.41% and 54.28%.

A new representation learning approach for credit data analysis

Representation learning has an important impact on the performance of machine learning methods and has been used to solve many distribution problems for numerous graphical and sequential mining tasks. While the distributions of credit data are very complex, the representations of such data are less studied. This study proposes a new representation learning approach based on a neural network called NyströmNet, which represents the credit data to benefit credit evaluation and sub-pattern analysis. The NyströmNet is developed to utilize the advantages of the Nyström method – a kernel approximation method in credit evaluation, yet overcomes its two limitations: distance distortions in kernel functions, and parameter tuning. The two main modules contained in NyströmNet, i.e., the Distance Metric Learning module and the Nyström module, can benefit each other and yield an overall optimum. Experiments using six real-life large-scale credit data showed that the AUC of the distance-based classifiers and the linear classifiers were improved by 2–11% and 2–14% with the newly generated distributions. The proposed approach also has certain practical advantages over traditional approaches because it is free from complex parameter tuning, consumes fewer memories, and is easy to utilize automatic differential frameworks such as PyTorch. The proposed approach is highly suitable for large-scale credit evaluation.

Few-Shot Class-Incremental SAR Target Recognition Based on Hierarchical Embedding and Incremental Evolutionary Network

It is difficult to realize effective synthetic aperture radar (SAR) automatic target recognition (ATR) in open scenarios because the ATR model cannot continuously learn from new classes with limited training samples. When adding new classes to the previously trained model, the capability of recognizing old classes may lose due to severe overfitting. To tackle this problem, a few-shot class-incremental SAR ATR method, namely hierarchical embedding and incremental evolutionary network (HEIEN), is proposed in this paper. Firstly, a hierarchical embedding network and a hybrid distance-based classifier is constructed for basic feature extraction and classification. Then, in order to obtain more accurate decision boundaries, an adaptive class-incremental learning module is designed to adjust the weights of classifiers in all tasks by collecting context information from the past to the present. Finally, a pseudo incremental training strategy is designed to enable effective model training with only a few samples. Experimental results on the MSTAR benchmark data set have illustrated that HEIEN performs well with remarkable advantages in few-shot class-incremental SAR ATR tasks.

Efficient cross-validation traversals in feature subset selection

Sparse and robust classification models have the potential for revealing common predictive patterns that not only allow for categorizing objects into classes but also for generating mechanistic hypotheses. Identifying a small and informative subset of features is their main ingredient. However, the exponential search space of feature subsets and the heuristic nature of selection algorithms limit the coverage of these analyses, even for low-dimensional datasets. We present methods for reducing the computational complexity of feature selection criteria allowing for higher efficiency and coverage of screenings. We achieve this by reducing the preparation costs of high-dimensional subsets {mathscr {O}}({n}m^2) to those of one-dimensional ones {mathscr {O}}(m^2). Our methods are based on a tight interaction between a parallelizable cross-validation traversal strategy and distance-based classification algorithms and can be used with any product distance or kernel. We evaluate the traversal strategy exemplarily in exhaustive feature subset selection experiments (perfect coverage). Its runtime, fitness landscape, and predictive performance are analyzed on publicly available datasets. Even in low-dimensional settings, we achieve approximately a 15-fold increase in exhaustively generating distance matrices for feature combinations bringing a new level of evaluations into reach.

The Distance Function Optimization for the Near Neighbors-Based Classifiers

Based on the analysis of conditions for a good distance function we found four rules that should be fulfilled. Then, we introduce two new distance functions, a metric and a pseudometric one. We have tested how they fit for distance-based classifiers, especially for the IINC classifier. We rank distance functions according to several criteria and tests. Rankings depend not only on criteria or nature of the statistical test, but also whether it takes into account different difficulties of tasks or whether it considers all tasks as equally difficult. We have found that the new distance functions introduced belong among the four or five best out of 23 distance functions. We have tested them on 24 different tasks, using the mean, the median, the Friedman aligned test, and the Quade test. Our results show that a suitable distance function can improve behavior of distance-based classification rules.

Classification and Recognition of Bilingual Text Using Graph Edit Distance Based Degree of Similarity

Objectives: Graph Edit distance-based classification and recognition method is introduced in this study for bilingual characters. Specifically, this method aims to classify characters first and then recognize them in the 2nd level. Methods: This study combines both exact graph matching and inexact graph matching techniques to achieve better Recognition. The exact graph matching technique classifies characters by considering the number of vertices and edges as features to classify. Inexact graph matching uses an algorithmic model to measure the degree of similarity in the length of edit operations. Findings: The proposed model can compute, if not optimal, sequences all the time, at least near-optimal edit sequence. The proposed model is based on dynamic programming. Edit distance is estimated by comparing preorder sequences of the respective binary trees. Finally, this model uses the loss function to recognize text accurately. To bring out the practicality of the proposed model, we present a case study on the classification of English alphabets and Kannada conjunct consonants. Novelty: (i) A hybrid approach for converting a large class classification problem into a small classification problem that takes advantage of both exact and inexact graph matching techniques. (ii) An algorithmic model is designed to represent each character as a unique string. (iii) Uses dynamic programming in the right places to get the optimal tree, and (iv) to convert the graph matching problem to a string matching problem reduces the time complexity of the problem. Keywords: Bilingual text; Character Recognition; Degree of Similarity; Graph Edit Distance; Graph Matching

An Accurate Method of Determining Attribute Weights in Distance-Based Classification Algorithms

Weight determination aims to determine the importance of different attributes; determining accurate weights can significantly improve the accuracy of classification and clustering. This paper proposes an accurate method for attribute weight determination. The method uses the distance from the sample point of each class to the class center point. It can minimize the weights and determines the attribute weights of the constraints through the objective function. In this paper, the attribute weights obtained by the exact solution are applied to the K-means clustering algorithm; three classic machine learning data sets, the iris data set, the wine data set, and the wheat seed data set, are clustered. Using the normalized mutual information as the evaluation index, a confusion matrix was established. Finally, the clustering results are visualized and compared with other methods to verify the effectiveness of the proposed method. The results show that this method improves the normalized mutual information by 0.11 and 0.08, respectively, compared with the unweighted and entropy weighted methods for iris clustering results. Furthermore, the performance on the wine data set is improved by 0.1, and the performance on the wheat seed data set is improved by 0.15 and 0.05.

Mapping paddy rice and rice phenology with Sentinel-1 SAR time series using a unified dynamic programming framework

AbstractMonitoring rice planting areas and their phenological phases is crucial for yield estimation and informed decision-making. This study proposed a unified method for mapping rice field and rice phenology with a dynamic time wrapping (DTW) distance-based classifier and its variant sub-DTW algorithm using Sentinel-1’s synthetic aperture radar (SAR) VH band. Field samplings were conducted for broad landcover types in one of the areas of interest (AOIs). We implemented a pixel-wisek-nearest neighbor classification model with DTW distance to identify paddy rice pixels. Standard rice phenological profiles of the SAR VH band were defined by ground monitoring of a sample rice field. Based on rice planting maps and the standard phenological profiles, rice phenological phases were estimated by pattern matching strategy with the sub-DTW algorithm. Experiments on six counties in Northeast China presented promising results. The overall producer and user accuracy reached 92.9 and 91.9% for rice mapping, respectively. The mean root mean square error (RMSE) for phenology estimation was 3.5 days. Rice planting and rice phenology maps were generated for the six AOIs. The phenological variances of the AOIs implied the effects of climate and rice cultivars on phenological development.

Mean curve length: An efficient feature for brainwave biometrics

Electroencephalography (EEG) as a biometric modality has gained considerable interest in recent years. Many state-of-the-art methods have focused on increasing the recognition accuracy. However, the more complex and manipulative the methods become, the less practical and generalized they are in real-life applications. In this study, we prioritized computational efficiency and evaluated the model performance. In this direction, we propose the mean curve length (MCL), a simple measure quantifying signal complexity, which is analytically and empirically related to the Katz fractal dimension. By merely being the average of the absolute value of the first-order difference of a signal, MCL is arguably the most computationally efficient feature that can be extracted from an EEG signal. In this paper, we utilized it for person identification and authentication on a large standard dataset comprising 109 subjects under the eyes-open (EO) and eyes-closed (EC) resting state conditions. We employed a Mahalanobis distance-based classifier both for identification and authentication tasks. Our results indicate that in addition to its simplicity and low computational cost, MCL provides a remarkably high individual distinction as well. Specifically, recognition accuracies were 99.4% (EO) and 98.8% (EC) for identification, and for authentication, equal error percentages of 6.33% (EO) and 10.50% (EC) were obtained. Our study offers a fast and accurate neural biometric recognition scheme promising especially for practical real-world and real-time applications. It further proves the effectiveness of nonlinear signal measures in individual discrimination, and promotes shifting the focus beyond the conventional brain oscillatory and connectivity measures commonly fostered in EEG-based biometrics literature.

Optimization of Fuzzy Support Vector Machine (FSVM) Performance by Distance-Based Similarity Measure Classification

This research aims to determine the maximum or minimum value of a Fuzzy Support Vector Machine (FSVM) Algorithm using the optimization function. SVM is considered as an effective method of data classification, as opposed to FSVM, which is less effective on large and complex data because of its sensitivity to outliers and noise. One of the techniques used to overcome this inefficiency is fuzzy logic with its ability to select the right membership function, which significantly affects the effectiveness of the FSVM algorithm performance. This research was carried out using the Gaussian membership function and the Distance-Based Similarity Measurement consisting of the Euclidean, Manhattan, Chebyshev, and Minkowsky distance methods. Subsequently, the optimization of the FSVM classification process was determined using four proposed FSVM models and normal SVM as comparison references. The results showed that the method tends to eliminate the impact of noise and enhance classification accuracy effectively. FSVM provides the best and highest accuracy value of 94% at a penalty parameter value of 1000 using the Chebyshev distance matrix. Furthermore, the model proposed will be compared to the performance evaluation model in preliminary studies (Xiao Kang et al., 2018). The result further showed that using FSVM with Chebyshev distance matrix and a Gaussian membership function provides a better performance evaluation value. Doi: 10.28991/HIJ-2021-02-04-02 Full Text: PDF

Distance-based directional depth classifiers: a robustness study

Contaminated training sets can highly affect the performance of classification rules. For this reason, robust supervised classifiers have been introduced. Amongst the many, this work focuses on depth-based classifiers, a class of methods which have been proven to enjoy some robustness properties. However, no robustness studies are available for them within a directional data framework. Here, their performance under some directional contamination schemes is evaluated. A comparison with the directional Bayes rule is also provided. Different directional specific contamination scenarios are introduced and discussed: antipodality and orthogonality of the contaminated distribution mean, and the directional mean shift outlier model.

Establishment of instrument operation qualification and routine performance qualification procedures for handheld near-infrared spectrometers used at different locations within a laboratory network

Quality assurance of finished pharmaceuticals is a necessity in ensuring the safety of consumers. There is a need for low-cost and portable rapid screening methods of pharmaceuticals in resource limited areas. Recent advances in technology have made handheld and low-cost diffuse reflectance spectrometers available to the public. While these handheld spectrometers offer advantages over benchtop spectrometers, the accuracy and repeatability must be assessed before these instruments can be used for quality assurance screening. Here, five handheld spectrometers of the same model were purchased, where an in-house installation qualification and operational qualification (IQOQ) was subsequently established for the instruments. Wavelength and photometric accuracy (and repeatability), spectroscopic noise, stray light, and bandpass were assessed between instruments. Results were found to be consistent between the spectrometers, passing IQOQ procedures, and were determined to be ready for field use. Once the handheld spectrometer’s performance was verified, a practical and low-cost daily performance verification was established using common high density polyethylene vial caps on location in South Africa, Thailand, and the United States. A Mahalanobis distance-based classifier found the five spectrometers to be in agreement.

SUBiNN: a stacked uni- and bivariate kNN sparse ensemble

Nearest Neighbor classification is an intuitive distance-based classification method. It has, however, two drawbacks: (1) it is sensitive to the number of features, and (2) it does not give information about the importance of single features or pairs of features. In stacking, a set of base-learners is combined in one overall ensemble classifier by means of a meta-learner. In this manuscript we combine univariate and bivariate nearest neighbor classifiers that are by itself easily interpretable. Furthermore, we combine these classifiers by a Lasso method that results in a sparse ensemble of nonlinear main and pairwise interaction effects. We christened the new method SUBiNN: Stacked Uni- and Bivariate Nearest Neighbors. SUBiNN overcomes the two drawbacks of simple nearest neighbor methods. In extensive simulations and using benchmark data sets, we evaluate the predictive performance of SUBiNN and compare it to other nearest neighbor ensemble methods as well as Random Forests and Support Vector Machines. Results indicate that SUBiNN often outperforms other nearest neighbor methods, that SUBiNN is well capable of identifying noise features, but that Random Forests is often, but not always, the best classifier.

Prototype generation in the string space via approximate median for data reduction in nearest neighbor classification

The k-nearest neighbor (kNN) rule is one of the best-known distance-based classifiers, and is usually associated with high performance and versatility as it requires only the definition of a dissimilarity measure. Nevertheless, kNN is also coupled with low-efficiency levels since, for each new query, the algorithm must carry out an exhaustive search of the training data, and this drawback is much more relevant when considering complex structural representations, such as graphs, trees or strings, owing to the cost of the dissimilarity metrics. This issue has generally been tackled through the use of data reduction (DR) techniques, which reduce the size of the reference set, but the complexity of structural data has historically limited their application in the aforementioned scenarios. A DR algorithm denominated as reduction through homogeneous clusters (RHC) has recently been adapted to string representations but as obtaining the exact median value of a set of string data is known to be computationally difficult, its authors resorted to computing the set-median value. Under the premise that a more exact median value may be beneficial in this context, we, therefore, present a new adaptation of the RHC algorithm for string data, in which an approximate median computation is carried out. The results obtained show significant improvements when compared to those of the set-median version of the algorithm, in terms of both classification performance and reduction rates.

Distance-based Classification and Regression Trees for the analysis of complex predictors in health and medical research.

Predicting patient outcomes based on patient characteristics and care processes is a common task in medical research. Such predictive features are often multifaceted and complex, and are usually simplified into one or more scalar variables to facilitate statistical analysis. This process, while necessary, results in a loss of important clinical detail. While this loss may be prevented by using distance-based predictive methods which better represent complex healthcare features, the statistical literature on such methods is limited, and the range of tools facilitating distance-based analysis is substantially smaller than those of other methods. Consequently, medical researchers must choose to either reduce complex predictive features to scalar variables to facilitate analysis, or instead use a limited number of distance-based predictive methods which may not fulfil the needs of the analysis problem at hand. We address this limitation by developing a Distance-Based extension of Classification and Regression Trees (DB-CART) capable of making distance-based predictions of categorical, ordinal and numeric patient outcomes. We also demonstrate how this extension is compatible with other extensions to CART, including a recently published method for predicting care trajectories in chronic disease. We demonstrate DB-CART by using it to expand upon previously published dose-response analysis of stroke rehabilitation data. Our method identified additional detail not captured by the previously published analysis, reinforcing previous conclusions. We also demonstrate how by combining DB-CART with other extensions to CART, the method is capable of making predictions about complex, multifaceted outcome data based on complex, multifaceted predictive features.

Efficient computation of counterfactual explanations and counterfactual metrics of prototype-based classifiers

The increasing use of machine learning in practice and legal regulations like EU’s GDPR cause the necessity to be able to explain the prediction and behavior of machine learning models. A prominent example of particularly intuitive explanations of AI models in the context of decision making are counterfactual explanations. Yet, it is still an open research problem how to efficiently compute counterfactual explanations for many models. In this contribution, we investigate how to efficiently compute counterfactual explanations for an important class of models, prototype-based classifiers such as learning vector quantization models. In particular, we derive specific convex and non-convex programs depending on the used metric. Typically counterfactual explanations deliver a feedback in terms of changes of the input features which lead to a different output – one application scenario is the link of these required changes to actionable items to change the desired outcome. Yet, rather than minimum changes of the input, it is interesting to address minimum changes of the model itself, which are required to lead to a different result. rather than a change of its inputs. We phrase this question as a counterfactual of the model prescription rather than the data points. We focus on distance-based classifiers (in particular learning vector quantization models), where model changes correspond to changes of metric parameters, and we develop efficient optimization techniques to generate such counterfactual metric changes depending on the chosen model.

PIECEWISE ANOMALY DETECTION USING MINIMAL LEARNING MACHINE FOR HYPERSPECTRAL IMAGES

Abstract. Hyperspectral imaging, with its applications, offers promising tools for remote sensing and Earth observation. Recent development has increased the quality of the sensors. At the same time, the prices of the sensors are lowering. Anomaly detection is one of the popular remote sensing applications, which benefits from real-time solutions. A real-time solution has its limitations, for example, due to a large amount of hyperspectral data, platform’s (drones or a cube satellite) constraints on payload and processing capability. Other examples are the limitations of available energy and the complexity of the machine learning models. When anomalies are detected in real-time from the hyperspectral images, one crucial factor is to utilise a computationally efficient method. The Minimal Learning Machine is a distance-based classification algorithm, which can be modified for anomaly detection. Earlier studies confirms that the Minimal learning Machine (MLM) is capable of detecting efficiently global anomalies from the hyperspectral images with a false alarm rate of zero. In this study, we will show that by using a carefully selected lower threshold besides the higher threshold of the variance, it is possible to detect local and global anomalies with the MLM. The downside is that the improved method is highly sensitive with the respect to the noise. Thus, the second aim of this study is to improve the MLM’s robustness with respect to noise by introducing a novel approach, the piecewise MLM. With the new approach, the piecewise MLM can detect global and local anomalies, and the method is significantly more robust with respect to noise than the MLM. As a result, we have an interesting, easy to implement and computationally light method which is suitable for remote sensing applications.