Probability Of Misclassification Research Articles

Estimating parameters of mixtures of multivariate [formula omitted]-populations and application to classification of observations

The problem of classifying an observation into mixtures of two multivariate t-distributions is studied when the location parameters and covariance matrices are unknown. For mixtures of two multivariate t-distributions, we propose classification rules based on the maximum likelihood estimators and Bayes estimators of the parameters. The maximum penalized likelihood estimators of the parameters are derived using some informative penalty function. We derive shrinkage estimators of the covariance matrices using a regularized parameter and propose the corresponding classification rule. The kernel density-based rule is proposed considering the diagonal matrix as a bandwidth parameter. A simulation study is carried out to compare the rules in terms of the expected probability of misclassification. Applications of the rules are described using real data sets arising in clinical studies and stock market.

Reducing bias due to misclassified exposures using instrumental variables

AbstractExposures are often misclassified in observational studies. Any analysis that does not make proper adjustments for misclassification may result in biased estimates of model parameters, resulting in distorted inference. Settings where a multicategory exposure variable has more than two nominal categories or where no validation data are available to assess misclassification probabilities are common in practice but seldom considered in the literature. This article presents a novel method of analyzing cohort data with a misclassified, multicategory exposure variable and a binary response variable that uses instrumental variables in lieu of a validation dataset. First, a sufficient condition is obtained for model identifiability. Then, methods for model estimation and inference are proposed after adopting a sufficient condition for identifiability. We consider a variational Bayesian inference procedure aided by automatic differentiation along with Markov chain Monte Carlo‐based computation. Operating characteristics of the proposed methods are assessed through simulation studies. For the purpose of illustration, the proposed Bayesian methods are applied to the U.S. breast cancer mortality data sampled from the Surveillance Epidemiology and End Results database, where reported treatment therapy is the misclassified multicategory exposure variable.

GPS Spoofing Detection by Neural Network Machine Learning

GPS today is ubiquitous. It provides real-time positioning, navigation, and timing (PNT) data for countless military and civilian users worldwide. Yet, a proliferation of GPS degrading and denying devices threatens GPS PNT capabilities. The biggest threat is spoofing, which deceives a GPS receiver into accepting false signals as genuine. Current antispoofing techniques are highly dependent on specific spoofing scenarios, choice of mathematical models, values of thresholds, and often require significant hardware and/or software modifications to existing GPS receivers. Efforts are also underway to replace GPS with alternate sources. However, no such sensor can match the availability, accuracy, and global utility of GPS. This article examines an alternate antispoofing method using neural networks. The supervised machine learning technique uses well-understood GPS observables such as pseudorange, carrier phase, Doppler shift, and carrier-to-noise density ratio to distinguish between authentic and spoofed signals. Network architecture, training sample size, number of hidden layers, distribution of neurons between hidden layers, and number of hidden neurons are examined. Results show that the proposed method is capable of detecting spoofing signals with a high probability of correct classification and a low probability of misclassification.

Random Fourier Feature-Based Deep Learning for Wireless Communications

Deep-learning (DL) has emerged as a powerful machine-learning technique for several problems encountered in generic wireless communications. Also, random Fourier Features (RFF) based DL has emerged as an attractive solution for several machine-learning problems; yet existing works lack rigorous analytical results to justify the viability of RFF based DL. To address this gap, we analytically quantify the viability of RFF based DL in this paper. Precisely, we present analytical proofs which show that the RFF based DL architectures have lower approximation-error and a lower probability of misclassification as compared to classical DL architectures for a fixed dataset-size. In addition, a new distribution-dependent RFF (DDRFF) is proposed to facilitate DL architectures with low training-complexity. The presented analytical contributions and the DDRFF are validated by relevant case-studies such as: a) line of sight (LOS)/non-line of sight (NLOS) classification, and b) message-passing based detection of low-density parity check (LDPC) codes over nonlinear visible light communication (VLC) channels. Especially, in the low training-data regime, the presented simulations depict significant performance gains for RFF based DL. Lastly, in all the presented simulations, it is observed that the proposed DDRFFs significantly outperform RFFs, which make them useful for potential machine-learning/DL applications for communication systems.

Detection Tolerant Black-Box Adversarial Attack Against Automatic Modulation Classification With Deep Learning

Advances in adversarial attack and defense technologies will enhance the reliability of deep learning (DL) systems spirally. Most existing adversarial attack methods make overly ideal assumptions, which creates the illusion that the DL system can be attacked simply and has restricted the further improvement on DL systems. To perform practical adversarial attacks, a detection tolerant black-box adversarial-attack (DTBA) method against DL-based automatic modulation classification (AMC) is presented in this article. In the DTBA method, the local DL model as a substitution of the remote target DL model is trained first. The training dataset is generated by an attacker, labeled by the target model, and augmented by Jacobian transformation. Then, the conventional gradient attack method is utilized to generate adversarial attack examples toward the local DL model. Moreover, before launching attack to the target model, the local model estimates the misclassification probability of the perturbed examples in advance and deletes those invalid adversarial examples. Compared with related attack methods of different criteria on public datasets, the DTBA method can reduce the attack cost while increasing the rate of successful attack. Adversarial attack transferability of the proposed method on the target model has increased by more than 20%. The DTBA method will be suitable for launching flexible and effective black-box adversarial attacks against DL-based AMC systems.

Multi-classification for high-dimensional data using probabilistic neural networks

Multi-classification tasks need sufficient information provided by the input data, whereas the input data lying in the high-dimensional space presents too sparse distributions to afford rich information, which creates trouble for multi-classification tasks. To address this, this paper proposed a probabilistic neural network approach. Firstly, the classification probability of high-dimensional data is calculated by using a novel Equation for Naïve Bayesian in order to reduce misclassification probability caused by the high dimension of the input data. Then, the calculated classification probability is fused into the cost function of the neural networks, forming the probabilistic neural networks. Results on high-dimensional real-world datasets and the National Surface Water Quality data provided by Environmental Monitoring Station, China show that the proposed method outperforms comparison methods in both classification ability and resisting the curse of dimensionality. We find that neural networks hardly obtain those advanced classification results upon a high-dimensional space because of the curse of dimensionality, but calculating classification probability of the input data can be confident to help neural networks to be resistance to the negative effects caused by high dimension of the input data.

Estimating species misclassification with occupancy dynamics and encounter rates: A semi‐supervised, individual‐level approach

Abstract Large‐scale, long‐term biodiversity monitoring is essential to conservation, land management and identifying threats to biodiversity. However, multispecies surveys are prone to various types of observation error, including false‐positive/false‐negative detection and misclassification, where a species is thought to have been encountered but not correctly identified. Previous methods assume an imperfect classifier produces species‐level classifications, but in practice, particularly with human observers, we may end up with extraspecific classifications including ‘unknown’, morphospecies designations and taxonomic identifications coarser than species. Disregarding these types of species misclassification in biodiversity monitoring datasets can bias estimates of ecologically important quantities such as demographic rates, occurrence and species richness. Here we present a joint classification‐occupancy model that accounts for species non‐detection and misclassification. Our framework accommodates extinction and colonization dynamics, allows for additional uncertain ‘morphospecies’ designations and makes use of individual specimens with known species identities in a semi‐supervised setting. We compare the performance of our model to a classification‐only model that discards information about occupancy and encounter rate. We illustrate our model with an empirical case study of the carabid beetle (Carabidae) community at the National Ecological Observatory Network Niwot Ridge Mountain Research Station, near Boulder, CO, USA. We also use simulations to evaluate model performance through validation metrics where varying fractions of the data are confirmed. The model supported imperfect classifier accuracy and favoured certain true species classifications strongly for some morphospecies. The model outperformed (e.g. precision) the reduced model that discarded occupancy information, and these differences were most pronounced for abundant species. Spatial and temporal dynamics from modelled occupancy and encounter rates may inform species misclassification probability, but this idea has not yet been tested. Our statistical framework explores this opportunity, and can be applied to datasets with imperfect species detection and classification, limited verification data and non‐species classifications.

Limit state Kriging modeling for reliability-based design optimization through classification uncertainty quantification

Reliability-based design optimization (RBDO) plays a vital role in considering the effect of uncertainties in the optimal design variables on the production reliability. Kriging-assisted RBDO methods can reduce the computational cost of conventional RBDO methods by replacing the time-consuming performance functions with Kriging models. Existing Kriging-assisted RBDO methods, however, are easy to fall into the low modeling efficiency issue or unsatisfied modeling accuracy issue because of the low utilization rate of sample resources. In this paper, an adaptive Kriging sampling strategy based on the Classification Uncertainty Quantification (KCUQ) was proposed. In KCUQ, the classification uncertainty of the Kriging model is sufficiently considered by (1) determining the new sample point based on the quantified misclassification probability and (2) checking the modeling accuracy based on the quantified number of misclassified random points. Moreover, KCUQ only updates the performance function with the largest classification error in each iteration such that all performance functions can be adaptively modeled based on their unique features. Two numerical case studies, vehicle side impact crashworthiness problem and the axle bridge turning parameters optimization application are used to demonstrate the performance of the proposed KCUQ method.

A robust scalar-on-function logistic regression for classification

Scalar-on-function logistic regression, where the response is a binary outcome and the predictor consists of random curves, has become a general framework to explore a linear relationship between the binary outcome and functional predictor. Most of the methods used to estimate this model are based on the least-squares type estimators. However, the least-squares estimator is seriously hindered by outliers, leading to biased parameter estimates and an increased probability of misclassification. This paper proposes a robust partial least squares method to estimate the regression coefficient function in the scalar-on-function logistic regression. The regression coefficient function represented by functional partial least squares decomposition is estimated by a weighted likelihood method, which downweighs the effect of outliers in the response and predictor. The estimation and classification performance of the proposed method is evaluated via a series of Monte Carlo experiments and a strawberry puree data set. The results obtained from the proposed method are compared favorably with existing methods.

Approximation of misclassification probabilities in linear discriminant analysis based on repeated measurements

The classification of observations based on repeated measurements performed on the same subject over a given period of time or under different conditions is a common procedure in many disciplines such as medicine, psychology and environmental studies. In this article repeated measurements follow an extended growth curve model and are classified using linear discriminant analysis. The aim of this article is to propose approximation for the misclassification probabilities in the linear discriminant function when the population means follow an extended growth curve structure. Using specific statistic relations we derive the approximation of misclassification probabilities for known and unknown covariance matrices. Finally, we perform a Monte Carlo simulation study to assess the accuracy of the developed results.

Fixed-Horizon Active Hypothesis Testing

Two active hypothesis testing problems are formulated. In these problems, the agent can perform a fixed number of experiments and then decide on one of the hypotheses or declare its experiments inconclusive. The first problem is an asymmetric formulation in which the objective is to minimize the probability of incorrectly declaring a particular hypothesis to be true while ensuring that the probability of correctly declaring that hypothesis is moderately high. This formulation is a generalization of the formulation in the Chernoff–Stein lemma to an active setting. The second problem is a symmetric formulation in which the objective is to minimize the misclassification probability while ensuring that the true hypothesis is declared conclusively with moderately high probability. For these problems, lower and upper bounds on the optimal misclassification probabilities are derived and these bounds are shown to be asymptotically tight. Novel experiment selection strategies are provided. It is shown that these strategies are asymptotically optimal and using numerical experiments, it is demonstrated that these strategies have a significantly better performance in the nonasymptotic regime.

A Binary Choice Model with Sample Selection and Covariate-Related Misclassification

Misclassification of a binary response variable and nonrandom sample selection are data issues frequently encountered by empirical researchers. For cases in which both issues feature simultaneously in a data set, we formulate a sample selection model for a misclassified binary outcome in which the conditional probabilities of misclassification are allowed to depend on covariates. Assuming the availability of validation data, the pseudo-maximum likelihood technique can be used to estimate the model. The performance of the estimator accounting for misclassification and sample selection is compared to that of estimators offering partial corrections. An empirical example illustrates the proposed framework.

Estimation of the average treatment effect on the treated with misclassified binary outcome

The estimation of the average treatment effect on the treated (ATT) plays an essential role when the effect of an intervention or a treatment on those participants who actually received it is the focus. However, the validity of conventional estimation methods relies on the precise measurement of variables. Misclassified category outcome variables may cause non‐negligible bias when estimating ATT. In this paper, under the assumption that the misclassification probability is homogeneous, we develop a bias‐corrected estimation method to consistently estimate ATT when internal validation data are available for a subgroup of the study population. We further derive a doubly robust estimator by augmenting the bias‐corrected estimator to provide protection against treatment model misspecification. Through simulation experiments and real data analysis, we demonstrate the satisfactory performance of the proposed estimators.

Multi-label learning via minimax probability machine

In this paper, we propose Minimax Probability Machine for Multi-label data classification and is termed as Multi-Label Minimax Probability Machine (MLMPM). Based on data mean and covariance information, MLMPM builds a classifier that minimizes an upper bound on the mis-classification probability of unseen future data. For capturing label correlation we have considered asymmetric co-occurrency matrix into the model. The proposed model has also been extended to non-linear settings using the Mercer Kernel trick. To accelerate the training procedure, iterative weighted least squares is used to train the underlying optimization model efficiently. Extensive experimental comparisons of our proposed method with related multi-label algorithms on synthetic as well as real world multi-label datasets, along with Amazon rainforest satellite images dataset, prove its efficacy.

An Incremental Learning Method Based on Dynamic Ensemble RVM for Intrusion Detection

Due to the dynamic changes of network data over time, static intrusion detection systems cannot adapt well to the behavioral characteristics of the input network data, resulting in reduced detection accuracy. In addition, continuous input data streams will bring huge challenges to resource storage and computing costs. Therefore, we propose an intrusion detection method of dynamic ensemble incremental learning (DEIL-RVM), and realize a dynamically adjusted ensemble intrusion detection model. In which a new overall misclassification probability weight value (OMPW) based on incremental set or data chunk is designed as the basis for updating the ensemble model, and it can be used to prune and replace the poor base component in the ensemble model. We presented a probabilistic decision function taking into account the posterior probability of each base RVM model dividing the sample into each category. The RVM with high sparsity is used as the base component to obtain the good balance between the accuracy, robustness and resource consumption, which can sacrifice less time and storage cost in ensemble incremental learning while achieving higher detection accuracy and stability in network data streams.

Community detection for multilayer weighted networks

Multilayer networks are used to encode multiple types of relations arising in complex systems and have received significant attention in recent years. Community detection in multilayer networks is an important issue in various fields; hence, stochastic block models have emerged as a popular probabilistic framework over the past decades. However, stochastic block models are suited to binary networks rather than weighted networks. A generalised stochastic block model is proposed herein to address multilayer sparse or dense weighted networks. A variational expectation-maximisation algorithm is derived to estimate the parameters of interest. In addition, an upper bound is derived for the probability of misclassification, which is governed by the Rényi divergence of order 12. Furthermore, our model is compared with four competing methods on synthetic networks. Finally, our approach is examined on financial markets and bicycle sharing systems.

Effect of passive metallic layers on muon energy estimation by means of deflection angle for muon scattering tomography: a comparative study based on GEANT4 simulations

In the tomographic configurations based on the muon scattering, the angular variation with respect to the kinetic energy indirectly brings forth the ability to coarsely predict the kinetic energy by using the deflection angle owing to the detector layers. Nevertheless, the angular deviation due to the detector components is expected to be minuscule in addition to a relatively high uncertainty in the case of the plastic scintillators. In the present study, we contrast our current tomographic prototype, which consists of the detector layers manufactured from polyvinyl toluene besides a detector accuracy of 1 mrad, with an alternative hodoscope scheme containing stainless steel layers by aiming to investigate the three-group energy structure. Initially, we determine the average deflection angles together with the corresponding standard deviations for our present setup as well as for the alternative scheme by means of the GEANT4 simulations. In the second place, we express a brace of misclassification probabilities founded on the standard deviations where the first procedure assumes a linear finite approximation, whereas the latter approach rests on a positively defined modified Gaussian distribution. Upon our simulation results, we demonstrate that the introduced stainless steel layers in the proposed hodoscope setup do not only serve to augment the average deflection angles, but they also diminish the misclassification probabilities, therewith reducing the classification uncertainty apart from an improved detection performance.

Exact Distribution-Free Hypothesis Tests for the Regression Function of Binary Classification via Conditional Kernel Mean Embeddings

In this paper we suggest two statistical hypothesis tests for the regression function of binary classification based on conditional kernel mean embeddings. The regression function is a fundamental object in classification as it determines both the Bayes optimal classifier and the misclassification probabilities. A resampling based framework is presented and combined with consistent point estimators of the conditional kernel mean map, in order to construct distribution-free hypothesis tests. These tests are introduced in a flexible manner allowing us to control the exact probability of type I error for any sample size. We also prove that both proposed techniques are consistent under weak statistical assumptions, i.e., the type II error probabilities pointwise converge to zero.

Weight Vector Tuning and Asymptotic Analysis of Binary Linear Classifiers

Unlike its intercept, a linear classifier's weight vector cannot be tuned by a simple grid search. Hence, this paper proposes weight vector tuning of a generic binary linear classifier through the parameterization of a decomposition of the discriminant by a scalar which controls the trade-off between conflicting informative and noisy terms. By varying this parameter, the original weight vector is modified in a meaningful way. Applying this method to a number of linear classifiers under a variety of data dimensionality and sample size settings reveals that the classification performance loss due to non-optimal native hyperparameters can be compensated for by weight vector tuning. This yields computational savings as the proposed tuning method reduces to tuning a scalar compared to tuning the native hyperparameter, which may involve repeated weight vector generation along with its burden of optimization, dimensionality reduction, etc., depending on the classifier. It is also found that weight vector tuning significantly improves the performance of Linear Discriminant Analysis (LDA) under high estimation noise. Proceeding from this second finding, an asymptotic study of the misclassification probability of the parameterized LDA classifier in the growth regime where the data dimensionality and sample size are comparable is conducted. Using random matrix theory, the misclassification probability is shown to converge to a quantity that is a function of the true statistics of the data. Additionally, an estimator of the misclassification probability is derived. Finally, computationally efficient tuning of the parameter using this estimator is demonstrated on real data.

Optimal Target Classification Using Frequency-Based Radar Waveform Design

In this article, we introduce a frequency-based waveform design to maximize binary target classification. The optimization problems can be solved via optimal solvers available in the literature. The presented formulation is applicable to the classification scenarios where the extended target frequency responses (TFRs) are complex, random, and normally distributed with unequal mean vectors. However, their covariance matrices are either identical or different. For the former, i.e., identical covariance matrices, the optimization problem consists of a cost function that has been obtained by deriving a closed-form expression of the probability of misclassification. For the latter, i.e., different covariance matrices, we present an optimization problem where the cost function has been obtained via the Fisher separation function. For this, we also present a closed-form expression for an approximate solution to the optimization problem. We show that the proposed solution achieves performance levels comparable to the exact solution of the optimization problem obtained via state-of-the-art optimization solvers while incurring low computational complexity. We expand on the two main scenarios by introducing clutter (i.e., signal-dependent interference) in the signal model and studying possible closed-form designs in extreme waveform energy levels. Simulations are conducted using synthetically generated data in addition to the civilian vehicle data from the Moving and Stationary Target Acquisition and Recognition (MSTAR) dataset, in order to validate the proposed method.