Distance-based Classification Research Articles

Light curve classification with DistClassiPy: A new distance-based classifier

The rise of synoptic sky surveys has ushered in an era of big data in time-domain astronomy, making data science and machine learning essential tools for studying celestial objects. While tree-based models (e.g. Random Forests) and deep learning models dominate the field, we explore the use of different distance metrics to aid in the classification of astrophysical objects. We developed DistClassiPy, a new distance metric based classifier. The direct use of distance metrics is unexplored in time-domain astronomy, but distance-based methods can help make classification more interpretable and decrease computational costs. In particular, we applied DistClassiPy to classify light curves of variable stars, comparing the distances between objects of different classes. Using 18 distance metrics on a catalog of 6,000 variable stars across 10 classes, we demonstrate classification and dimensionality reduction. Our classifier meets state-of-the-art performance but has lower computational requirements and improved interpretability. Additionally, DistClassiPy can be tailored to specific objects by identifying the most effective distance metric for that classification. To facilitate broader applications within and beyond astronomy, we have made DistClassiPy open-source and available at https://pypi.org/project/distclassipy/.

Real-time hybrid facial expression analysis model using quantum distance-based classifier and classical artificial neural networks: QUADCANN

Real-time hybrid facial expression analysis model using quantum distance-based classifier and classical artificial neural networks: QUADCANN

SKZC: self-distillation and k-nearest neighbor-based zero-shot classification

Zero-shot learning represents a formidable paradigm in machine learning, wherein the crux lies in distilling and generalizing knowledge from observed classes to novel ones. The objective is to identify unfamiliar objects that were not included in the model’s training, leveraging learned patterns and knowledge from previously encountered categories. As a crucial subtask of open-world object detection, zero-shot classification can also provide insights and solutions for this field. Despite its potential, current zero-shot classification models often suffer from a performance gap due to limited transfer ability and discriminative capability of learned representations. In pursuit of advancing the subpar state of zero-shot object classification, this paper introduces a novel model for image classification which can be applied to object detection, namely, self-distillation and k-nearest neighbor-based zero-shot classification method. First, we employ a diffusion detector to identify potential objects in images. Then, self-distillation and distance-based classifiers are used for distinguishing unseen objects from seen classes. The k-nearest neighbor-based cluster heads are designed to cluster the unseen objects. Extensive experiments and visualizations were conducted on publicly available datasets on the efficacy of the proposed approach. Precisely, our model demonstrates performance improvement of over 20% compared to contrastive clustering. Moreover, it achieves a precision of 0.910 and a recall of 0.842 on CIFAR-10 datasets, a precision of 0.737, and a recall of 0.688 on CIFAR-100 datasets for the macro average. Compared to a more recent model (SGFR), our model realized improvements of 10.9%, 13.3%, and 7.8% in Sacc, Uacc, and H metrics, respectively. This study aims to introduce fresh ideas into the domain of zero-shot image classification, and it can be applied to open-world object detection tasks. Our code is available at https://www.github.com/CmosWolf1/Code_implementation_for_paper_SKZC.

Few-Shot Class-Incremental Learning for Medical Time Series Classification.

Continuously analyzing medical time series as new classes emerge is meaningful for health monitoring and medical decision-making. Few-shot class-incremental learning (FSCIL) explores the classification of few-shot new classes without forgetting old classes. However, little of the existing research on FSCIL focuses on medical time series classification, which is more challenging to learn due to its large intra-class variability. In this paper, we propose a framework, the Meta self-Attention Prototype Incrementer (MAPIC) to address these problems. MAPIC contains three main modules: an embedding encoder for feature extraction, a prototype enhancement module for increasing inter-class variation, and a distance-based classifier for reducing intra-class variation. To mitigate catastrophic forgetting, MAPIC adopts a parameter protection strategy in which the parameters of the embedding encoder module are frozen at incremental stages after being trained in the base stage. The prototype enhancement module is proposed to enhance the expressiveness of prototypes by perceiving inter-class relations using a self-attention mechanism. We design a composite loss function containing the sample classification loss, the prototype non-overlapping loss, and the knowledge distillation loss, which work together to reduce intra-class variations and resist catastrophic forgetting. Experimental results on three different time series datasets show that MAPIC significantly outperforms state-of-the-art approaches by 27.99%, 18.4%, and 3.95%, respectively.

Prediction and classification model of hornet sighting report in Washington state based on deep learning

In recent years, the precise control of pests has become a concern for multiple departments. However, the accurate identification of misclassifications in eyewitness reports has remained a significant challenge. This study presents a comprehensive prediction and classification model designed for hornet sighting reports in Washington State. Leveraging deep learning, image analysis, and geographic location processing, the model aims to address the challenges associated with the accurate classification of reported sightings as positive, negative, or unverified. The methodology integrates transfer learning with ResNet and employs data augmentation techniques to enhance image-based predictions. The use of PyTorch facilitates neural network construction and training, leading to notable improvements in accuracy, especially in recognizing negative cases. Furthermore, geographic location processing introduces an innovative dimension, utilizing spatial information for distance-based classification. By combining the sigmoid function with geographical distances, predictions are refined, particularly for negative samples. An auxiliary function enhances predictions for samples lacking images. The practical prediction approach integrates image and location data, producing comprehensive results. The model evaluation demonstrates its efficacy through extensive data analysis. The significance of this study lies in its contribution to filling research gaps within related fields and supporting effective pest management, particularly in response to the threat posed by the Asian giant hornet. The obtained comprehensive model can accurately classify future collected eyewitness reports to guide relevant departments in their prevention and control strategies.

On the Search for Potentially Anomalous Traces of Cosmic Ray Particles in Images Acquired by Cmos Detectors for a Continuous Stream of Emerging Observational Data.

In this paper we propose the method for detecting potential anomalous cosmic ray particle tracks in big data image dataset acquired by Complementary Metal-Oxide-Semiconductors (CMOS). Those sensors are part of scientific infrastructure of Cosmic Ray Extremely Distributed Observatory (CREDO). The use of Incremental PCA (Principal Components Analysis) allowed approximation of loadings which might be updated at runtime. Incremental PCA with Sequential Karhunen-Loeve Transform results with almost identical embedding as basic PCA. Depending on image preprocessing method the weighted distance between coordinate frame and its approximation was at the level from 0.01 to 0.02 radian for batches with size of 10,000 images. This significantly reduces the necessary calculations in terms of memory complexity so that our method can be used for big data. The use of intuitive parameters of the potential anomalies detection algorithm based on object density in embedding space makes our method intuitive to use. The sets of anomalies returned by our proposed algorithm do not contain any typical morphologies of particle tracks shapes. Thus, one can conclude that our proposed method effectively filter-off typical (in terms of analysis of variance) shapes of particle tracks by searching for those that can be treated as significantly different from the others in the dataset. We also proposed method that can be used to find similar objects, which gives it the potential, for example, to be used in minimal distance-based classification and CREDO image database querying. The proposed algorithm was tested on more than half a million (570,000+) images that contains various morphologies of cosmic particle tracks. To our knowledge, this is the first study of this kind based on data collected using a distributed network of CMOS sensors embedded in the cell phones of participants collaborating within the citizen science paradigm.

Statelets extraction-guided contrastive learning for detection of control performance degradation with varying degrees

Detection of control performance degradation (CPD) in time series plays a vital role in modern industry. However, most CPD detection methods tend to neglect the following recognitions: 1) control performance is often assessed over small segments of the time series rather than the long-term entire one, and 2) CPD may exhibit different degrees in practice, resulting in the time-varying data distribution problem. These two considerations necessitate the development of algorithms that can extract information-rich small segments, defined as statelets, to discriminate the control performance with varying degrees. To this end, a Statelets Extraction-based Contrastive Learning (SECL) model is proposed for the detection of CPD. First, a varying-length statelet generation algorithm is developed to split the original multivariate time series into multiple statelet candidates with different time lengths, which can avoid the problem of information-rich small segments being obscured by the long-term trend of the time series. A contrastive learning encoder is then constructed to map statelet candidates into a uniform feature space with the designed triplet loss, keeping features of similar statelets tight and features of dissimilar statelets far away. In this way, we can extract discriminative feature candidates to distinguish the control performance with varying degrees. An information entropy-based selection strategy is proposed to choose information-rich statelet features that can best identify different control performances. After that, a statelets distance-based classifier is designed by computing the match distance between features of a testing sample and the extracted statelet features. The proposed SECL model focuses on the extracted statelets that can significantly reveal changes related to control performance rather than comparing the entire time series, which is more accurate and robust in detecting CPD with varying degrees. The effectiveness of the proposed method is illustrated through a real-world thermal power plant process. The experiment results show that SECL can capture the discriminative features and achieve prominent detection performance.

Metric learning for monotonic classification: turning the space up to the limits of monotonicity

This paper presents, for the first time, a distance metric learning algorithm for monotonic classification. Monotonic datasets arise in many real-world applications, where there exist order relations in the input and output variables, and the outputs corresponding to ordered pairs of inputs are also expected to be ordered. Monotonic classification can be addressed through several distance-based classifiers that are able to respect the monotonicity constraints of the data. The performance of distance-based classifiers can be improved with the use of distance metric learning algorithms, which are able to find the distances that best represent the similarities among each pair of data samples. However, learning a distance for monotonic data has an additional drawback: the learned distance may negatively impact the monotonic constraints of the data. In our work, we propose a new model for learning distances that does not corrupt these constraints. This methodology will also be useful in identifying and discarding non-monotonic pairs of samples that may be present in the data due to noise. The experimental analysis conducted, supported by a Bayesian statistical testing, demonstrates that the distances obtained by the proposed method can enhance the performance of several distance-based classifiers in monotonic problems.

EDIR: Efficient Distributed Image Retrieval of Novel Objects in Mobile Networks

Crowdsourcing data collection from a network of mobile devices is useful in various applications. Mobile devices store a large amount of visual data that can aid in different application scenarios. Trained Convolutional Neural Networks (CNNs) can be deployed on mobile devices to be used in searching for objects of interest. Querying for novel objects, for which models have not been trained yet, presents some unique challenges. When novel objects are queried, new models must be trained and distributed to all edge devices. In this paper, we propose an efficient method and a system, called EDIR, which enables answering these queries while taking into account the bandwidth limitations encountered in wireless networks, as well as the limited energy and computational power on mobile devices. Through extensive experimentation, we show that using distance-based classifiers, specifically those relying on the Cosine distance, leads to more efficient utilization of network resources by reducing the number of false positives. We perform analysis that enables the requester to tune the parameters of interest before issuing the query, and validate our theoretical results. EDIR reduces the amount of transferred data by more than 45% compared to other approaches while simultaneously achieving a good F1 score.

Analysis of Machine Learning Classifiers for Speaker Identification: A Study on SVM, Random Forest, KNN, and Decision Tree

This study investigates the performance of machine learning classifiers in the domain of speaker identification, a pivotal component of modern digital security systems. With the burgeoning integration of voice-activated interfaces in technology, the demand for accurate and reliable speaker identification is paramount. This research provides a comprehensive comparison of four widely used classifiers: Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), and Decision Tree (DT). Utilizing the LibriSpeech dataset, known for its diversity of speakers and recording conditions, we extracted Mel-frequency cepstral coefficients (MFCCs) to serve as features for training and evaluating the classifiers. Each model's performance was assessed based on precision, recall, F1-score, and accuracy. The results revealed that RF outperformed all other classifiers, achieving near-perfect metrics, indicative of its robustness and generalizability for speaker identification tasks. KNN also demonstrated high performance, suggesting its suitability for applications where rapid execution and interpretability are critical. Conversely, SVM and DT, while yielding moderate and lower performances respectively, highlighted the necessity for further optimization. These findings underscore the effectiveness of ensemble and distance-based classifiers in handling complex patterns for speaker differentiation. The study not only guides the selection of appropriate classifiers for speaker identification but also sets the stage for future research, which could explore hybrid models and the impact of dataset variability on performance. The insights from this analysis contribute significantly to the field, providing a benchmark for developing advanced speaker identification systems

High performance Legionella pneumophila source attribution using genomics-based machine learning classification.

Fundamental to effective Legionnaires' disease outbreak control is the ability to rapidly identify the environmental source(s) of the causative agent, Legionella pneumophila. Genomics has revolutionized pathogen surveillance, but L. pneumophila has a complex ecology and population structure that can limit source inference based on standard core genome phylogenetics. Here, we present a powerful machine learning approach that assigns the geographical source of Legionnaires' disease outbreaks more accurately than current core genome comparisons. Models were developed upon 534 L. pneumophila genome sequences, including 149 genomes linked to 20 previously reported Legionnaires' disease outbreaks through detailed case investigations. Our classification models were developed in a cross-validation framework using only environmental L. pneumophila genomes. Assignments of clinical isolate geographic origins demonstrated high predictive sensitivity and specificity of the models, with no false positives or false negatives for 13 out of 20 outbreak groups, despite the presence of within-outbreak polyclonal population structure. Analysis of the same 534-genome panel with a conventional phylogenomic tree and a core genome multi-locus sequence type allelic distance-based classification approach revealed that our machine learning method had the highest overall classification performance-agreement with epidemiological information. Our multivariate statistical learning approach maximizes the use of genomic variation data and is thus well-suited for supporting Legionnaires' disease outbreak investigations.IMPORTANCEIdentifying the sources of Legionnaires' disease outbreaks is crucial for effective control. Current genomic methods, while useful, often fall short due to the complex ecology and population structure of Legionella pneumophila, the causative agent. Our study introduces a high-performing machine learning approach for more accurate geographical source attribution of Legionnaires' disease outbreaks. Developed using cross-validation on environmental L. pneumophila genomes, our models demonstrate excellent predictive sensitivity and specificity. Importantly, this new approach outperforms traditional methods like phylogenomic trees and core genome multi-locus sequence typing, proving more efficient at leveraging genomic variation data to infer outbreak sources. Our machine learning algorithms, harnessing both core and accessory genomic variation, offer significant promise in public health settings. By enabling rapid and precise source identification in Legionnaires' disease outbreaks, such approaches have the potential to expedite intervention efforts and curtail disease transmission.

Quantum variational distance-based centroid classifier

Quantum machine learning is a rapidly growing field with the potential to surpass classical methods and offer novel solutions to learning problems. Supervised classification, a prominent pattern recognition task, has attracted significant attention for improving existing methods using quantum techniques. Although previous quantum algorithms for distance-based classification have shown promise, they are not compatible with current Noisy Intermediate-Scale Quantum (NISQ) devices and cannot handle multiclass problems. To address these limitations, we present the quantum Variational Distance-based Centroid Classifier (qVDCC), a hybrid approach that combines the Quantum One-Class Classifier (QOCC) and Parameterized Quantum Circuits (PQCs). This approach greatly improves the feasibility of multiclass classification on NISQ devices, as it eliminates the need for a label qubit and uses a single-qubit measurement, regardless of data size. Our simulations on seven benchmark datasets using both noisy and noise-free simulations show that qVDCC is competitive with the k-nearest neighbors algorithm, while providing exponential data compression and efficient quantum data processing.

A 3d convolutional neural network autoencoder for predicting solvent configuration changes in condensed phase biomass reactions

Self-supervised 3d CNN autoencoder combined with a distance-based classifier is trained on first principles simulations to predict the extent of solvent configuration changes during reactive chemical transformations.

Improved Design for Hardware Implementation of Graph-Based Large Margin Classifiers for Embedded Edge Computing.

The number of connected embedded edge computing Internet of Things (IoT) devices has been increasing over the years, contributing to the significant growth of available data in different scenarios. Thereby, machine learning algorithms arise to enable task automation and process optimization based on those data. However, due to some learning methods' computational complexity implementing geometric classifiers, it is a challenge to map these on embedded systems or devices with limited resources in size, processing, memory, and power, to accomplish the desired requirements. This hampers the applicability of these methods to complex industrial embedded edge applications. This work evaluates strategies to reduce classifiers' implementation costs based on the CHIP-clas model, independent of hyperparameter tuning and optimization algorithms. The proposal aims to evaluate the tradeoff between numerical precision and model performance and analyze the hardware implementations of a distance-based classifier. Two 16 -b floating-point formats were compared to the 32 -b floating-point precision implementation. Also, a new hardware architecture was developed and then compared to the state-of-the-art reference. The results indicate that the model is robust to low precision computation, providing statistically equivalent results compared to the baseline model, also pointing out statistically equivalent performance and a global speed-up factor of approx 4.39 in processing time.

A hybrid few-shot multiple-instance learning model predicting the aggressiveness of lymphoma in PET/CT images

Background and objectivePatients with aggressive non-Hodgkin lymphoma (NHL) undergo distinct therapy strategies compared with indolent NHL patients. However, it is challenging to estimate NHL aggressiveness based on visual inspection of positron emission tomography (PET) or computed tomography (CT) images. Since diffuse large B-cell lymphoma (DLBCL) and Follicular lymphoma (FL) are the most typical and dominant aggressive and indolent NHL, respectively, this study aims to develop an artificial-intelligence-enabled model to distinguish DLBCL from FL in PET/CT images as the first step to tackle this challenge. MethodsWe propose a hybrid few-shot multiple-instance learning model to predict the aggressiveness of the NHL. First, rotation-based self-supervision learning (SSL) has been employed to train the encoder on a large-scale, publicly available CT image dataset. Second, hybrid instance-level features are obtained for each NHL lesion by combining deep features with the radiomics features from both PET and CT modalities. Third, instance-level features are transformed into bag-level (or patient-level) representations. Finally, bag-level representations are fed into a distance-based classifier through few-shot learning to predict NHL aggressiveness. ResultsOur model achieves an accuracy of 0.751 ± 0.008, a sensitivity of 0.787 ± 0.012, a specificity of 0.715 ± 0.013, an F1-score of 0.753 ± 0.009, and an area under the curve (AUC) of 0.795 ± 0.009 at the bag level. It outperforms the typical counterparts that use the radiomic features, random forest for feature selection, and support vector machines (SVMs) as classifiers. The three counterparts yield accuracies of 0.714 ± 0.023, 0.705 ± 0.008, and 0.698 ± 0.008, respectively. Moreover, settings of the SSL training dataset (Deep lesion) and task (rotation), hybrid CT and radiomic PET features, the pool-layer strategy of maximum, and distance-based classifier generate the best model. ConclusionsA hybrid few-shot multiple-instance learning model can predict lymphoma aggressiveness in PET/CT images and could be a potential tool for determining therapy strategies. Hybrid features and the combination of SSL, few-shot learning, and weakly supervised learning are the two powerful pillars of the model, and these can be expanded to other medical applications with limited samples and incomplete annotations.

Building Multiple Classifier Systems Using Linear Combinations of Reduced Graphs

Despite great efforts done in research in the last decades, the classification of general graphs, i.e., graphs with unconstrained labeling and structure, remains a challenging task. Due to the inherent relational structure of graphs it is difficult, or even impossible, to apply standard pattern recognition methods to graphs to achieve high recognition accuracies. Common methods to solve the non-trivial problem of graph classification employ graph matching in conjunction with a distance-based classifier or a kernel machine. In the present paper, we address the specific task of graph classification by means of a novel framework that uses information acquired from a broad range of reduced graph subspaces. Our novel approach can be roughly divided into three successive steps. In the first step, differently reduced graphs are created out of the original graphs relying on node centrality measures. In the second step, we compute the graph edit distance between each reduced graph and all the other graphs of the corresponding graph subspace. Finally, we linearly combine the distances in the third step and feed them into a distance-based classifier to obtain the final classification result. On six graph data sets, we empirically confirm that the proposed multiple classifier system directly benefits from the combined distances computed in the various graph subspaces.

On the Distribution of Muscle Signals: A Method for Distance-Based Classification of Human Gestures.

We investigate the distribution of muscle signatures of human hand gestures under Dynamic Time Warping. For this we present a k-Nearest-Neighbors classifier using Dynamic Time Warping for the distance estimate. To understand the resulting classification performance, we investigate the distribution of the recorded samples and derive a method of assessing the separability of a set of gestures. In addition to this, we present and evaluate two approaches with reduced real-time computational cost with regards to their effectiveness and the mechanics behind them. We further investigate the impact of different parameters with regards to practical usability and background rejection, allowing fine-tuning of the induced classification procedure.

Causal interaction in high frequency turbulence at the biosphere–atmosphere interface: Structural behavior

High-frequency (e.g., 10 Hz) eddy covariance measurements are typically used to estimate fluxes at the land–atmosphere interface at timescales of 15–60 min. These multivariate data contain information about the interdependency at high frequency between the interacting variables such as wind, humidity, temperature, and CO2. We use data at 10 Hz from an eddy covariance instrument located at 25 m above agricultural land in the Midwestern US, which offers an opportunity to move beyond the traditional spectral analyses to explore causal dependency among variables. In this study, we quantify the structure of inter-dependencies of interacting variables at high frequency represented by a directed acyclic graph (DAG). We compare DAGs to investigate changes in structural differences in causal interactions. We then apply a distance-based classification and k-means clustering approach to identify the evolution of the causal structure represented by a DAG. Our method selects an unbiased number of clusters of similar structures and characterizes the similarities and differences between them. We explore a range of dynamic behavior using data from a clear sky day and during a solar eclipse in 2017. Our results show well-defined clusters of similar causal dependencies as the system evolves. Our approach provides a methodological framework to understand how causal dependence in turbulence manifests in high-frequency data when represented through a DAG.

A Representation of Cloth States based on a Derivative of the Gauss Linking Integral

Robotic manipulation of cloth is a complex task because of the infinite-dimensional shape-state space of textiles, which makes their state estimation very difficult. In this paper we introduce the dGLI Cloth Coordinates, a finite low-dimensional representation of cloth states that allows us to efficiently distinguish a large variety of different folded states, opening the door to efficient learning methods for cloth manipulation planning and control. Our representation is based on a directional derivative of the Gauss Linking Integral and allows us to represent spatial as well as planar folded configurations in a consistent and unified way. The proposed dGLI Cloth Coordinates are shown to be more accurate in the representation of cloth states and significantly more sensitive to changes in grasping affordances than other classic shape distance methods. Finally, we apply our representation to real images of a cloth, showing that with it we can identify the different states using a distance-based classifier.

Comparison of Distance Function in K-Nearest Neighbor Algorithm to Predict Prospective Customers in Term Deposit Subscriptions

Data mining is often used to analysis of the big data to obtain new useful information that will be used in the future. One of the best algorithms in data mining is K-Nearest Neighbor (KKN). K-NN classifier is a distance-based classification algorithm. The distance function is a core component in measuring the distance or similarity between the tested data and the training data. Various measure of distance function exist make this a topic of kind literature problems to determining the best distance function for the performance of the K-NN classifier. This study aims to compare which distance function produces the best K-NN performance. The distance function to be compared is the Manhattan distance and Minkowski distance. The application of K-NN classifier using bank dataset about predict prospective customers in Term Deposit Subscriptions. This study show that Minkowski distance on K-NN algorithm achieved the best result compared to Manhattan distance. Minkowski distance with power p = 1.5 produces an accuracy rate of 88.40% when the K value is 7. Thus, performance of K-NN algorithm using Minkowski distance (p=1,5, K=7) is best algorithm in predicting prospective costumers in Term Deposit Subscription