Nearest Mean Classifier Research Articles

Non-exemplar Domain Incremental Object Detection via Learning Domain Bias

Domain incremental object detection (DIOD) aims to gradually learn a unified object detection model from a dataset stream composed of different domains, achieving good performance in all encountered domains. The most critical obstacle to this goal is the catastrophic forgetting problem, where the performance of the model improves rapidly in new domains but deteriorates sharply in old ones after a few sessions. To address this problem, we propose a non-exemplar DIOD method named learning domain bias (LDB), which learns domain bias independently at each new session, avoiding saving examples from old domains. Concretely, a base model is first obtained through training during session 1. Then, LDB freezes the weights of the base model and trains individual domain bias for each new incoming domain, adapting the base model to the distribution of new domains. At test time, since the domain ID is unknown, we propose a domain selector based on nearest mean classifier (NMC), which selects the most appropriate domain bias for a test image. Extensive experimental evaluations on two series of datasets demonstrate the effectiveness of the proposed LDB method in achieving high accuracy on new and old domain datasets. The code is available at https://github.com/SONGX1997/LDB.

Supervised versus unsupervised approaches to classification of accelerometry data.

Sophisticated animal-borne sensor systems are increasingly providing novel insight into how animals behave and move. Despite their widespread use in ecology, the diversity and expanding quality and quantity of data they produce have created a need for robust analytical methods for biological interpretation. Machine learning tools are often used to meet this need. However, their relative effectiveness is not well known and, in the case of unsupervised tools, given that they do not use validation data, their accuracy can be difficult to assess. We evaluated the effectiveness of supervised (n = 6), semi-supervised (n = 1), and unsupervised (n = 2) approaches to analyzing accelerometry data collected from critically endangered California condors (Gymnogyps californianus). Unsupervised K-means and EM (expectation-maximization) clustering approaches performed poorly, with adequate classification accuracies of <0.8 but very low values for kappa statistics (range: -0.02 to 0.06). The semi-supervised nearest mean classifier was moderately effective at classification, with an overall classification accuracy of 0.61 but effective classification only of two of the four behavioral classes. Supervised random forest (RF) and k-nearest neighbor (kNN) machine learning models were most effective at classification across all behavior types, with overall accuracies >0.81. Kappa statistics were also highest for RF and kNN, in most cases substantially greater than for other modeling approaches. Unsupervised modeling, which is commonly used for the classification of a priori-defined behaviors in telemetry data, can provide useful information but likely is instead better suited to post hoc definition of generalized behavioral states. This work also shows the potential for substantial variation in classification accuracy among different machine learning approaches and among different metrics of accuracy. As such, when analyzing biotelemetry data, best practices appear to call for the evaluation of several machine learning techniques and several measures of accuracy for each dataset under consideration.

Attention correlated appearance and motion feature followed temporal learning for activity recognition

&lt;span lang="EN-US"&gt;Recent advances in deep neural networks have been successfully demonstrated with fairly good accuracy for multi-class activity identification. However, existing methods have limitations in achieving complex spatial-temporal dependencies. In this work, we design two stream fusion attention (2SFA) connected to a temporal bidirectional gated recurrent unit (GRU) one-layer model and classified by prediction voting classifier (PVC) to recognize the action in a video. Particularly in the proposed deep neural network (DNN), we present 2SFA for capturing appearance information from red green blue (RGB) and motion from optical flow, where both streams are correlated by proposed fusion attention (FA) as the input of a temporal network. On the other hand, the temporal network with a bi-directional temporal layer using a GRU single layer is preferred for temporal understanding because it yields practical merits against six topologies of temporal networks in the UCF101 dataset. Meanwhile, the new proposed classifier scheme called PVC employs multiple nearest class mean (NCM) and the SoftMax function to yield multiple features outputted from temporal networks, and then votes their properties for high-performance classifications. The experiments achieve the best average accuracy of 70.8% in HMDB51 and 91.9%, the second best in UCF101 in terms of 2DConvNet for action recognition.&lt;/span&gt;

A Signal Processing Framework for the Detection of Abnormal Cardiac Episodes.

Cardiologists rely on the long duration Holter electrocardiogram (ECG) recordings in general for assessment of abnormal episodes and such process found to be tedious and time consuming. An automatic abnormal cardiac episode detection algorithm is the need of the hour that needs to be optimized to reduce the manual burden. The current study presents a signal processing framework with a cross-database to detect abnormal episodes in long-term ECG signals. The data was pre-processed to remove power line interference and baseline drift using basis pursuit sparsely decomposed tunable-Q wavelet transform (BPSD-TQWT). A total of 44 features of time domain, frequency domain, and time-frequency domain characteristics were extracted from the ECG signal. This proposed work tested classification performance with support vector machine (SVM), K-nearest neighbour (KNN), decision tree, naïve Bayes, the nearest mean classifier, and the nearest root mean square classifiers. The trained models with open-source data were used to predict the abnormal episodes from the proprietary database and vice versa. Finally, the performance was analysed via recall rate, specificity, precision, F1-score, and accuracy. Among six classification models, SVM performed best. With an open-source database, the SVM model achieved 95.01% accuracy, and detected the abnormal episodes from proprietary database with an accuracy of 99.31%. In addition, with the proprietary database SVM model classified the normal-abnormal cardiac episodes with an accuracy of 99.89% and detected the abnormal episodes from proprietary database with an accuracy of 92.51%. When the performance results were compared with the literature, it was observed that the proposed framework performed well. As a result, the proposed framework could be used in an autonomous diagnosis system.

Classification of Fruits Based on Shape and Color using Combined Nearest Mean Classifiers

Fruit classification is an important task in many agriculture industry. The fruit classification system can be used to identify the types and prices of fruit. Manual classification of fruit is not efficient for large amount of fruits. The advancement of information technology has made possible fruit classification be done by a machine. This research aims to propose a fruit classification methodology based on shape and color. To reduce the effect of lighting variability a color normalization is carried out prior to feature extraction. The color features used in this research are mean and standard deviation. The shape features are area, perimeter, and compactness. The classification of an unknown fruit is carried out using the nearest mean classifier. The method developed in this research is tested using 12 classes of fruits where each class is represented by a number of samples. The experimental results show that the method proposed in this research provides an accuracy of 95.83% for two samples per class and 100% for three samples per class. Experiment on small training samples has been conducted to evaluate the performance of the proposed combined nearest mean classifiers and results obtained showed that the technique was able to provide good accuracy.

Signal Processing Framework for the Detection of Ventricular Ectopic Beat Episodes.

The Holter monitor captures the electrocardiogram (ECG) and detects abnormal episodes, but physicians still use manual cross-checking. It takes a considerable time to annotate a long-term ECG record. As a result, research continues to be conducted to produce an effective automatic cardiac episode detection technique that will reduce the manual burden. The current study presents a signal processing framework to detect ventricular ectopic beat (VEB) episodes in long-term ECG signals of cross-database. The proposed study has experimented with the cross-database of open-source and proprietary databases. The ECG signals were preprocessed and extracted the features such as pre-RR interval, post-RR interval, QRS complex duration, QR slope, and RS slope from each beat. In the proposed work, four models such as support vector machine, k-means nearest neighbor, nearest mean classifier, and nearest RMS (NRMS) classifiers were used to classify the data into normal and VEB episodes. Further, the trained models were used to predict the VEB episodes from the proprietary database. NRMS has reported better performance among four classification models. NRMS has shown the classification accuracy of 98.68% and F1-score of 94.12%, recall rate of 100%, specificity of 98.53%, and precision of 88.89% with an open-source database. In addition, it showed an accuracy of 99.97%, F1-score of 94.54%, recall rate of 98.62%, specificity of 99.98%, and precision of 90.79% to detect the VEB cardiac episodes from the proprietary database. Therefore, it is concluded that the proposed framework can be used in the automatic diagnosis system to detect VEB cardiac episodes.

Robust Hybrid Classification Methods and Applications

The sample mean classifier, such as the nearest mean classifier (NMC) and the Bayes classifier, is not robust due to the influence of outliers. Enhancing the robust performance of these methods may result in vital information loss due to weighting or data deletion. The focus of this study is to develop robust hybrid univariate classifiers that do not rely on data weighting or deletion. The following data transformation methods, such as the least square approach (LSA) and linear prediction approach (LPA), are applied to estimate the parameters of interest to achieve the objectives of this study. The LSA and LPA estimates are applied to develop two groups of univariate classifiers. We further applied the predicted estimates from the LSA and LPA methods to develop four hybrid classifiers. These classifiers are applied to investigate whether cattle horn and base width length could be used to determine cattle gender. We also used these classification methods to determine whether shapes could classify banana variety. The NMC, LSA, LPA, and hybrid classifiers showed that cattle gender could be determined using horn length and base width measurement. The analysis further revealed that shapes could determine banana variety. The comparative results using the two data sets demonstrated that all the methods have over 90% performance prediction accuracy. The findings affirmed that the performance of the NMC, LSA, LPA, and the hybrid classifiers satisfy the data-dependent theory and are suitable for classifying agricultural products. Therefore, the proposed methods could be applied to perform classification tasks efficiently in many fields of study.

Online continual learning in image classification: An empirical survey

Online continual learning for image classification studies the problem of learning to classify images from an online stream of data and tasks, where tasks may include new classes (class incremental) or data nonstationarity (domain incremental). One of the key challenges of continual learning is to avoid catastrophic forgetting (CF), i.e., forgetting old tasks in the presence of more recent tasks. Over the past few years, a large range of methods and tricks have been introduced to address the continual learning problem, but many have not been fairly and systematically compared under a variety of realistic and practical settings.To better understand the relative advantages of various approaches and the settings where they work best, this survey aims to (1) compare state-of-the-art methods such as Maximally Interfered Retrieval (MIR), iCARL, and GDumb (a very strong baseline) and determine which works best at different memory and data settings as well as better understand the key source of CF; (2) determine if the best online class incremental methods are also competitive in the domain incremental setting; and (3) evaluate the performance of 7 simple but effective tricks such as the ”review” trick and the nearest class mean (NCM) classifier to assess their relative impact. Regarding (1), we observe that iCaRL remains competitive when the memory buffer is small; GDumb outperforms many recently proposed methods in medium-size datasets and MIR performs the best in larger-scale datasets. For (2), we note that GDumb performs quite poorly while MIR – already competitive for (1) – is also strongly competitive in this very different (but important) continual learning setting. Overall, this allows us to conclude that MIR is overall a strong and versatile online continual learning method across a wide variety of settings. Finally for (3), we find that all tricks are beneficial, and when augmented with the “review” trick and NCM classifier, MIR produces performance levels that bring online continual learning much closer to its ultimate goal of matching offline training. Our codes are available at https://github.com/RaptorMai/online-continual-learning.

Disease Classification and Prediction using Ensemble Machine Learning Classification Algorithm

In today’s scenario, disease prediction plays an important role in medical field. Early detection of diseases is essential because of the fast food habits and life. In my previous study for predicting diseases using radiology test report , and to classify the disease as positive or negative three classifiers Naïve Bayes (NB), Support Vector Machine (SVM) and Modified Extreme Learning Machine (MELM was used to increase the accuracy of results. To increase the efficiency of predicting the disease and to find which disease pricks the society, ensemble machine learning algorithm is used. The huge data from the healthcare industry were preprocessed., categorized and analyzed to find out and predict which patient to be treated and given priority and which hits the society the most. Ensemble machine learning’s popularity in the medical industry is due to a variety of factors the Classifiers used are K Nearest Neighbors, Nearest Mean Classifier, Mean Feature Voting Classifier, KDtree KNN, Random Forest. To reduce the manual processes in medical field automating these processes has become important. Electronic medical records and significant advances in health care have given an opportunity to make find out which patients need to be given more importance. Several methodologies and techniques were used to preprocess the data in order to meet the study’ requirements. To improve the performance of machine learning algorithms, feature selections were made using Tabu search. When ensemble prediction is combined with the Random Forest algorithm as the combiner, the results are more reliable. The aim of this study is to create a system to classify Medical records whether it is diseased or not and find out which disease rate has increased. This research will help the society to an individual to get treated easily and take preventive measures to avoid diseases.

ROBUST K-NEAREST NEIGHBOR CLASSIFIER AND NEAREST MEAN CLASSIFIER BASED INFORMATIVE KNOWLEDGE DISTILLATION FACE RECOGNITION

Human Face identification procedures have made tremendous progress in the last decade. Nevertheless, identifying faces with incomplete impediment is as yet challenging for the present face identifiers, and is very much required within certifiable application programs regarding reconnaissance and protection. Though great examination exertion has been dedicated to creating face de-impediment techniques, the greater part of them can just function admirably under obliged conditions. In this manuscript is proposed a Robust K-NNC (K-Nearest Neighbor Classifier) and NMC(Nearest Mean Classifier ) (RKNNC-NMC) prototype to efficiently reestablish incompletely occluded faces even in nature. This model comprises of two-stream, first introduced to perceive high-resolution faces and goal corrupted appearances with a student stream and a teacher stream, separately. A Teacher stream is signified by a Complex RKNNC-NMC for the sake of high-exactness recognition, and the student stream is signified by an a lot more straightforward RKNNC-NMC for low-unpredictability recognition. Broad examinations on synthetic and real datasets datasets of countenances with impediment plainly show the viability of RKNNC-NMC in eliminating various kinds of impediment in one’s face at different locations. The suggested technique additionally gives better behaviour gain than other de-occlusion strategies in advancing recognition execution through partially-occluded faces.

Prediction of Late-Onset Sepsis in Preterm Infants Using Monitoring Signals and Machine Learning.

Prediction of late-onset sepsis (onset beyond day 3 of life) in preterm infants, based on multiple patient monitoring signals 24 hours before onset. Continuous high-resolution electrocardiogram and respiration (chest impedance) data from the monitoring signals were extracted and used to create time-interval features representing heart rate variability, respiration, and body motion. For each infant with a blood culture-proven late-onset sepsis, a Cultures, Resuscitation, and Antibiotics Started Here moment was defined. The Cultures, Resuscitation, and Antibiotics Started Here moment served as an anchor point for the prediction analysis. In the group with controls (C), an "equivalent crash moment" was calculated as anchor point, based on comparable gestational and postnatal age. Three common machine learning approaches (logistic regressor, naive Bayes, and nearest mean classifier) were used to binary classify samples of late-onset sepsis from C. For training and evaluation of the three classifiers, a leave-k-subjects-out cross-validation was used. Level III neonatal ICU. The patient population consisted of 32 premature infants with sepsis and 32 age-matched control patients. No interventions were performed. For the interval features representing heart rate variability, respiration, and body motion, differences between late-onset sepsis and C were visible up to 5 hours preceding the Cultures, Resuscitation, and Antibiotics Started Here moment. Using a combination of all features, classification of late-onset sepsis and C showed a mean accuracy of 0.79 ± 0.12 and mean precision rate of 0.82 ± 0.18 3 hours before the onset of sepsis. Information from routine patient monitoring can be used to predict sepsis. Specifically, this study shows that a combination of electrocardiogram-based, respiration-based, and motion-based features enables the prediction of late-onset sepsis hours before the clinical crash moment.

A scalable network intrusion detection system towards detecting, discovering, and learning unknown attacks

Network intrusion detection systems (IDSs) based on deep learning have reached fairly accurate attack detection rates. But these deep learning approaches usually have been performed in a closed-set protocol that only known classes appear in training are considered during classification, the existing IDSs will fail to detect the unknown attacks and misclassify them as the training known classes, hence are not scalable. Furthermore, these IDSs are not efficient for updating the deep detection model once new attacks are discovered. To address those problems, we propose a scalable IDS towards detecting, discovering, and learning unknown attacks, it has three components. Firstly, we propose the open-set classification network (OCN) to detect unknown attacks, OCN based on the convolutional neural network adopts the nearest class mean (NCM) classifier, two new loss are designed to jointly optimize it, including Fisher loss and maximum mean discrepancy (MMD) loss. Subsequently, the semantic embedding clustering method is proposed to discover the hidden unknown attacks from all unknown instances detected by OCN. Then we propose the incremental nearest cluster centroid (INCC) method for learning the discovered unknown attacks through updating the NCM classifier. Extensive experiments on KDDCUP’99 dataset and CICIDS2017 dataset indicate that our OCN outperforms the state-of-the-art comparison methods in detecting multiple types of unknown attacks. Our experiments also verify the feasibility of the semantic embedding clustering method and INCC in discovering and learning unknown attacks.

Gene selection ensembles and classifier ensembles for medical diagnosis

Summary The usefulness of combining methods is examined using the example of microarray cancer data sets, where expression levels of huge numbers of genes are reported. Problems of discrimination into two groups are examined on three data sets relating to the expression of huge numbers of genes. For the three examined microarray data sets, the cross-validation errors evaluated on the remaining half of the whole data set, not used earlier for the selection of genes, were used as measures of classifier performance. Common single procedures for the selection of genes—Prediction Analysis of Microarrays (PAM) and Significance Analysis of Microarrays (SAM)—were compared with the fusion of eight selection procedures, or of a smaller subset of five of them, excluding SAM or PAM. Merging five or eight selection methods gave similar results. Based on the misclassification rates for the three examined microarray data sets, for any examined ensemble of classifiers, the combining of gene selection methods was not superior to single PAM or SAM selection for two of the examined data sets. Additionally, the procedure of heterogeneous combining of five base classifiers—k-nearest neighbors, SVM linear and SVM radial with parameter c=1, shrunken centroids regularized classifier (SCRDA) and nearest mean classifier—proved to significantly outperform resampling classifiers such as bagging decision trees. Heterogeneously combined classifiers also outperformed double bagging for some ranges of gene numbers and data sets, but merging is generally not superior to random forests. The preliminary step of combining gene rankings was generally not essential for the performance for either heterogeneously or homogeneously combined classifiers.

Fluorescence photo-bleaching of urine and its applicability in oral cancer diagnosis.

Photo-stability of urine is of crucial importance for the applicability of fluorescence spectroscopy of urine samples for diagnosis of cancer. We report the results of a detailed study on fluorescence photo-bleaching of human urine samples. We also present the results of a preliminary investigation on evaluation of the applicability of photo-bleaching characteristics of urine for discriminating patients with oral cancer from healthy volunteers. The time-lapse fluorescence induced by continuous shining of 405 nm radiation from a diode laser was recorded from the urine samples obtained from 18 patients with oral cancer as well as from 22 healthy volunteers with history of no known major illness in the past two months. The integrated fluorescence intensity (ΣI), calculated for each spectrum, was found to decrease with time till a point after which no further decrease was observed. Further, while significant differences were observed in the spectra of cancerous patients and healthy volunteers, these differences were found to be varying with time till the intensities of the observed fluorescence spectra corresponding to the two categories of urine samples became stable. The curve, generated by plotting ΣI vs. time, was found to be best fitted (R2 > 0.95) with a double-exponential decay function. The photo-bleaching constants, obtained from curve-fitting, were found to have statistically significant differences corresponding to the urine samples of cancerous patients and healthy volunteers. A classification algorithm developed based on nearest-mean classifier (NMC) and applied on the photo-bleaching constants in leave-one-subject-out cross-validation mode was found to provide a sensitivity and specificity of up to ∼ 86% in discriminating the two categories of urine samples.

Robust Feature-Based Automated Multi Viewhuman Action Recognition System Using Machine Learning

Automated human action Recognition has the potential to play an important role in Public security. In this project it compares three practical, reliable and generics systems for multiview video based human action recognition namely the nearest classifier, Gaussian mixture model classifier and nearest mean classifier.

Biometric human recognition system based on ECG

The ECG (electrocardiogram) is an emerging technology for biometric human identification. In this paper, the performance of an ECG biometric recognition system is evaluated. Signal processing techniques are utilized to extract the ECG features. In preprocessing stage, digital filters eliminate the noises and hence improve the signal to noise ratio. The process of ventricular complex (QRS Complex) detection depends on Pan and Tompkins approach that achieves an efficient QRS detection, and hence enhancing the feature extraction process. The main classifiers applied to the extracted features are Neural Network (NN), Fuzzy Logic (FL), Nearest Mean Classifier (NMC), Linear Discriminant Analysis (LDA), and Euclidean Distance (ED) are utilized to classify QRS fragments. ECG of an unknown subject is acquired; the classifiers are applied to wavelet coefficient features set between the unknown subject and all enrolled subjects. The Performance of the different approaches is evaluated by utilizing Sensitivity, Specificity, and efficiency, EER (Equal Error Rate) and ROC (Receiver Operating Characteristic) curve. The experiments are conducted on 112 individuals MIT-BIH database and the accuracy is up to 98.99%.

Quantum-inspired minimum distance classification in a biomedical context

We propose an application of a quantum-inspired version of the Nearest Mean Classifier (NMC) (G. Sergioli, E. Santucci, L. Didaci, J. A. Miszczak and R. Giuntini, A quantum inspired version of the NMC classifier, Soft Comput. 22(3) (2018) 691. G. Sergioli, G. M. Bosyk, E. Santucci and R. Giuntini, A quantum-inspired version of the classification problem, Int. J. Theo. Phys. 56(12) (2017) 3880. E. Santucci and G. Sergioli, Classification problem in a quantum framework, in quantum foundations, probability and information, Proc. Quantum and Beyond Conf., 13–16 June 2016, Vaxjo, Sweden, A. Khrennikov and T. Bourama, Springer-Berlin, Germany, 2018 (in press, 2018). E. Santucci, Quantum minimum distance classifier, Entropy 19(12) (2017) 659.) to a biomedical context. In particular, we benchmark the performances of such a quantum-variant of NMC against NMC and other (nonlinear) classifiers with respect to the problem of classifying the probability of survival for patients affected by idiopathic pulmonary fibrosis (IPF).

An Improved Flexible Partial Histogram Bayes Learning Algorithm

&lt;em&gt;This paper presents a proposed supervised classification technique namely flexible partial histogram Bayes (fPHBayes) learning algorithm. In our previous work, partial histogram Bayes (PHBayes) learning algorithm showed some advantages in the aspects of speed and accuracy in classification tasks. However, its accuracy declines when dealing with small number of instances or when the class feature distributes in wide area. In this work, the proposed fPHBayes solves these limitations in order to increase the classification accuracy. fPHBayes was analyzed and compared with PHBayes and other standard learning algorithms like first nearest neighbor, nearest subclass mean, nearest class mean, naive Bayes and Gaussian mixture model classifier. The experiments were performed using both real data and synthetic data considering different number of instances and different variances of Gaussians. The results showed that fPHBayes is more accurate and flexible to deal with different number of instances and different variances of Gaussians as compared to PHBayes.&lt;/em&gt;

A Nonnegative Locally Linear KNN model for image recognition

In this paper, we investigate the non-zero representation coefficients of Locally Linear KNN (LLKNN) and propose a nonnegative extension of LLKNN (NLLKNN) model for image recognition, where representation coefficients are restricted to be nonnegative to avoid meaningless and unreasonable negative coefficients. A multiplicative iterative algorithm with proof of convergence is proposed to solve the proposed NLLKNN model. Then NLLKNN based classifier (NLLKNNC) and Nonnegative Locally Linear Nearest Mean Classifier (NLLNMC) are proposed. We also investigate the robustness of NLLKNNC and NLLNMC to noises and occlusions. The effectiveness of the proposed methods is evaluated on several image recognition tasks such as scene recognition and face recognition. Extensive experimental results show that the proposed algorithm converges very fast and the proposed methods outperform some representative image recognition methods.

Robust Feature-Based Automated Multi-View Human Action Recognition System

Automated human action recognition has the potential to play an important role in public security, for example, in relation to the multiview surveillance videos taken in public places, such as train stations or airports. This paper compares three practical, reliable, and generic systems for multiview video-based human action recognition, namely, the nearest neighbor classifier, Gaussian mixture model classifier, and the nearest mean classifier. To describe the different actions performed in different views, view-invariant features are proposed to address multiview action recognition. These features are obtained by extracting the holistic features from different temporal scales which are modeled as points of interest which represent the global spatial-temporal distribution. Experiments and cross-data testing are conducted on the KTH, WEIZMANN, and MuHAVi datasets. The system does not need to be retrained when scenarios are changed which means the trained database can be applied in a wide variety of environments, such as view angle or background changes. The experiment results show that the proposed approach outperforms the existing methods on the KTH and WEIZMANN datasets.