Neyman-Pearson Research Articles

Neyman-Pearson Multi-class Classification via Cost-sensitive Learning

Most existing classification methods aim to minimize the overall misclassification error rate. However, in applications such as loan default prediction, different types of errors can have varying consequences. To address this asymmetry issue, two popular paradigms have been developed: the Neyman-Pearson (NP) paradigm and the cost-sensitive (CS) paradigm. Previous studies on the NP paradigm have primarily focused on the binary case, while the multi-class NP problem poses a greater challenge due to its unknown feasibility. In this work, we tackle the multi-class NP problem by establishing a connection with the CS problem via strong duality and propose two algorithms. We extend the concept of NP oracle inequalities, crucial in binary classifications, to NP oracle properties in the multi-class context. Our algorithms satisfy these NP oracle properties under certain conditions. Furthermore, we develop practical algorithms to assess the feasibility and strong duality in multi-class NP problems, which can offer practitioners the landscape of a multi-class NP problem with various target error levels. Simulations and real data studies validate the effectiveness of our algorithms. To our knowledge, this is the first study to address the multi-class NP problem with theoretical guarantees. The proposed algorithms have been implemented in the R package npcs, which is available on CRAN.

Spectrum sensing using 16-QAM and 32-QAM modulation techniques at different signal-to-noise ratio: a performance analysis

<p>Spectrum sensing techniques are implemented for effective use of spectrum resources in the cognitive-radio. In this research work attempt has been made for the performance of energy detector with cooperative spectrum sensing using double dynamic threshold on the MATLAB software. Additive white gaussian noise is used and the frequency range between 54 MHz to 862 MHz is considered with wideband. Findings in receiver operative curve have been observed to analyze probability-of-detection (𝑃𝑑) under different values of probability-of-false alarm (𝑃𝑓) condition and diverse ranges of signal to noise ratio with different number of samples of input signal. Presence and absence of primary user has been marked by using a hypothetical model based on Neyman pearson approach. From the results, it has been observed that more the number of sample values, better is the probability-of-detection (𝑃𝑑) value as achieved for 32-QAM signal as compared to 16-QAM signal. Also, better results have been witnessed at -9db signal to noise ratio value as compared to -15db and -20db. This work provides almost 10% of enhancement in the probability of detection at -9db signal to noise ratio for 16-QAM modulated signal as compared to the existing model where the implementation of energy detector spectrum sensing technique through simulink model.</p>

Attack detection and fault‐tolerant control of interconnected cyber‐physical systems against simultaneous replayed time‐delay and false‐data injection attacks

AbstractNowadays, interconnected cyber‐physical systems (CPSs) are widely used with increasing deployments of Industrial Internet of Things (IIoT) applications. Other than operating properly under system uncertainties, CPSs should be secured under unwanted adversaries. To mark such challenges, this paper proposes the solution of secure decentralized robust control for uncertain CPSs under replayed time‐delay and false‐data injection attacks altogether. Potentially, considered attacks can force the whole system to instability and crash. Three challenges are addressed, and solutions are presented: (1) model non‐linearity and uncertainties, (2) existing simultaneous time‐delay and potential false‐data injection attacks with skew probability density functions, and (3) requirement to use real‐time attack detection. Thus, a novel, robust control method to deal with thwart attacks on a closed‐loop control system is proposed to provide the system's trustworthiness. Additionally, novel attack detection methodologies are presented to detect these advanced attacks rapidly based on statistical methods such as Spearman's correlation coefficient, Neyman–Pearson (NP) error classification, and trend analysis. Ultimately, the proposed novel attack detection and robust control protocol are verified and evaluated in real‐time.

Blind Physical-Layer Authentication Based on Composite Radio Sample Characteristics

The promising physical-layer authentication (PLA) scheme enjoys low computational complexity and provides a lightweight solution for the wireless transmission security problem. However, the ideal but impractical assumptions are made on the priori knowledge in the traditional PLA schemes, which fail because the priori knowledge can be interfered by the impersonation attacks. Hence, a blind PLA scheme featured by the composite radio sample characteristics, active monitoring slots, and blind hypotheses tests is proposed in this paper. In particular, the intrinsic location-specific channel response integrated with the transmitter-specific signal knowledge is sampled during the active monitoring slots and is used to conduct minimum error (ME) and Neyman-Pearson (NP) hypothesis tests. The authentication reliability is enhanced significantly by jointly using the ME and NP criteria. For both the stationary and the non-stationary signal cases, our theoretical analysis shows that the temporal channel variations, the spatial channel correlations and the spectral bandwidth contribute to improve the authentication reliability. Our simulation results not only demonstrate the effectiveness of the proposed PLA scheme, but also indicate the possible scenarios in which ME outperforms NP, and NP outperforms ME.

Optimal Configuration of Array Elements for Hybrid Distributed PA-MIMO Radar System Based on Target Detection

This paper establishes a hybrid distributed phased array multiple-input multiple-output (PA-MIMO) radar system model to improve the target detection performance by combining coherent processing gain and spatial diversity gain. First, the radar system signal model and array space configuration model for the PA-MIMO radar are established. Then, a novel likelihood ratio test (LRT) detector is derived based on the Neyman–Pearson (NP) criterion in a fixed noise background. It can jointly optimize the coherent processing gain and spatial diversity gain of the system by implementing subarray level and array element level optimal configuration at both receiver and transmitter ends in a uniform blocking manner. On this basis, three typical optimization problems are discussed from three aspects, i.e., the detection probability, the effective radar range, and the radar system equipment volume. The approximate closed-form solutions of them are constructed and solved by the proposed quantum particle swarm optimization-based stochastic rounding (SR-QPSO) algorithm. Through the simulations, it is verified that the proposed optimal configuration of the hybrid distributed PA-MIMO radar system offers substantial improvements compared to the other typical radar systems, detection probability of 0.98, and an effective range of 1166.3 km, which significantly improves the detection performance.

Data-driven transmit weight matrix design method for target detection of the TS-FDA-MIMO radar

Recently, reinforcement learning (RL) has attracted more and more attention in research, and this method is implemented in different fields as an optimizer. In this paper, we apply RL to design the transmit weight matrix so as to obtain the optimal target detection performance, i.e., maximum detection probability. Firstly, the data model of transmit subaperturing frequency diverse array multiple-input multiple-output (TS-FDA-MIMO) radar is proposed. Next, we utilize the Neyman-Pearson (NP) criterion to deduce the detection probability of the TS-FDA-MIMO radar, and the transmit weight matrix design optimization problem is proposed. In addition, we construct an adaptive transmit weight matrix design framework based on the maximum detection probability and introduce the deep deterministic policy gradient (DDPG) algorithm of RL to the optimization controller in the proposed framework. The memory bank is used to store position information of the motive target, and the prior information is used in off-line training of the actor and critic networks. After network training, the current target state information is input into the RL network to obtain the output transmit weight matrix, and the optimization of the transmit weight matrix is completed. Finally, the simulation results verify the effectiveness of the proposed adaptive transmit weight matrix design framework based on RL.

On the effects of cloud water content on passive microwave snowfall retrievals

The Bayesian passive microwave retrievals of snowfall often rely on mathematical matching of the observed vectors of brightness temperature with an a priori database of precipitation profiles and their corresponding brightness temperatures. Mathematical proximity does not necessarily lead to consistent retrievals due to limited information content of passive microwave observations. This paper defines imposter (genuine) vectors of brightness temperature as those that are mathematically close but physically inconsistent (consistent) and characterizes them through the Silhouette Coefficient (SC) analysis. The Neyman–Pearson (NP) hypothesis testing is used to separate the imposter and genuine brightness temperatures based on their associated values of cloud ice (IWP) and liquid water path (LWP), given by coincidences of CloudSat Profiling Radar (CPR) and the Global Precipitation Measurement (GPM) Microwave Imager (GMI). The study determines thresholds for IWP and LWP that allow optimal identification of imposter brightness temperatures of non-snowing and snowing clouds, which can mislead the passive microwave retrieval algorithms to falsely detect or miss the snowfall events. It is demonstrated that emission signal of supercooled liquid water in snowing clouds can lead to improved passive microwave retrieval of snowfall and conditioning the retrievals to the cloud IWP and LWP can result in marginal correction of the snowfall detection probability; however, reduce the probability of false alarm by 6%–8% over sea ice and open oceans.

Fast copula-based fusion of correlated decisions for distributed radar detection

Distributed sensor networks with multiple antenna arrays are widely utilized for radar detection systems. When observations of network nodes are correlated, an optimal copula-based fusion (termed CBF) rule is designed for correlated binary decisions under the Neyman–Pearson (NP) framework. However, the CBF rule needs to determine the weights of all decisions in combination. Therefore, the high computational complexity of the CBF rule is unavoidable as the number of nodes increases. In this paper, we propose a fast copula-based fusion (F-CBF) algorithm for a multi-node radar system. The correlation between node decisions is simplified at the fusion center (FC) by taking logarithms of the weighted sum of node decisions. Specifically, the node decisions are weighted by a constant associated with the number of nodes. Thus the proposed method does not need to exhaust the weights of all combinations. Simulation results demonstrate that the proposed algorithm can efficiently reduce computational complexity. The detection performance of the proposed F-CBF method is significantly better than the independent detection method, although it is slightly inferior to the CBF method.

Joint Antenna Placement and Power Allocation for Target Detection in a Distributed MIMO Radar Network

Radar network configuration and power allocation are of great importance in military applications, where the entire surveillance area needs to be searched under resource budget constraints. To pursue the joint antenna placement and power allocation (JAPPA) optimization, this paper develops a JAPPA strategy to improve target detection performance in a widely distributed multiple-input and multiple-output (MIMO) radar network. First, the three variables of the problem are incorporated into the Neyman–Pearson (NP) detector by using the antenna placement optimization and the Lagrange power allocation method. Further, an improved iterative greedy dropping heuristic method based on a two-stage local search is proposed to solve the NP-hard issues of high-dimensional non-linear integer programming. Then, the sum of the weighted logarithmic likelihood ratio test (LRT) function is constructed as optimization criteria for the JAPPA approach. Numerical simulations and the theoretical analysis confirm the superiority of the proposed algorithm in terms of achieving effective overall detection performance.

Low-Complexity Distributed Detection With One-Bit Memory Under Neyman–Pearson Criterion

We consider a multi-stage distributed detection scenario, where $n$ sensors and a fusion center (FC) are deployed to accomplish a binary hypothesis test. At each time stage, local sensors generate binary messages, assumed to be spatially and temporally independent given the hypothesis, and then upload them to the FC for global detection decision making. We suppose a one-bit memory is available at the FC to store its decision history and focus on developing iterative fusion schemes. We first visit the detection problem of performing the Neyman-Pearson (N-P) test at each stage and give an optimal algorithm, called the oracle algorithm, to solve it. Structural properties and limitation of the fusion performance in the asymptotic regime are explored for the oracle algorithm. We notice the computational inefficiency of the oracle fusion and propose a low-complexity alternative, for which the likelihood ratio (LR) test threshold is tuned in connection to the fusion decision history compressed in the one-bit memory. The low-complexity algorithm greatly brings down the computational complexity at each stage from $O(4^n)$ to $O(n)$. We show that the proposed algorithm is capable of converging exponentially to the same detection probability as that of the oracle one. Moreover, the rate of convergence is shown to be asymptotically identical to that of the oracle algorithm. Finally, numerical simulations and real-world experiments demonstrate the effectiveness and efficiency of our distributed algorithm.

Asymmetric Error Control Under Imperfect Supervision: A Label-Noise-Adjusted Neyman–Pearson Umbrella Algorithm

Label noise in data has long been an important problem in supervised learning applications as it affects the effectiveness of many widely used classification methods. Recently, important real-world applications, such as medical diagnosis and cybersecurity, have generated renewed interest in the Neyman–Pearson (NP) classification paradigm, which constrains the more severe type of error (e.g., the Type I error) under a preferred level while minimizing the other (e.g., the Type II error). However, there has been little research on the NP paradigm under label noise. It is somewhat surprising that even when common NP classifiers ignore the label noise in the training stage, they are still able to control the Type I error with high probability. However, the price they pay is excessive conservativeness of the Type I error and hence a significant drop in power (i.e., 1 - Type II error). Assuming that domain experts provide lower bounds on the corruption severity, we propose the first theory-backed algorithm that adapts most state-of-the-art classification methods to the training label noise under the NP paradigm. The resulting classifiers not only control the Type I error with high probability under the desired level but also improve power.

Identification of active users for grant-free massive connectivity in large scale antenna systems

Conventional grant-based random access scheme is inappropriate to massive Internet of Things (IoT) connectivity since massive devices results in large number of collisions. This is unacceptable for the low latency requirement in 5 G and future networks. It is also not possible to assign orthogonal pilot sequences to all users to perform user activity detection (UAD) due to the massive number of devices and limited channel coherence time. In this paper, a novel grant-free (GF) UAD scheme is proposed with extremely low complexity and latency in an IoT network with a massive number of users. We exploit multiple antennas at the base station (BS) to produce spatial filtering by a fixed beamforming network (FBN), there then the inter-beam interference can be mitigated. Moreover, intra-beam interference is removed in temporal domain by orthogonal multiple access (OMA) technology. Joint UAD and multiuser detection (MUD) is realized by a bank of spatial-temporal matched filters at BS. The proposed method is efficient and the complexity is much less than the existing compressed sensing (CS)-based GF non-orthogonal multiple access (GFNOMA) algorithms. Performances of the proposed method is extensively analyzed in terms of the successful activity detection rate (SADR) as well as the Receiver operating characteristic (ROC) based on Neyman-Pearson (NP) decision rule. Numerical results demonstrate that it is comparable to the recently proposed iterative Maximum Likelihood (ML) algorithm, yet the computation load of the proposed scheme is extensively reduced.

AEC Classifier: A Tree-Based Classifier with Error Control for Medical Disease Diagnosis and Other Applications

In medical disease diagnosis, the cost of a false negative could greatly outweigh the cost of a false positive. This is because the former could cost a life, whereas the latter may only cause medical costs and stress to the patient. The unique nature of this problem highlights the need of asymmetric error control for binary classification applications. In this domain, traditional machine learning classifiers may not be ideal as they do not provide a way to control the number of false negatives below a certain threshold. This paper proposes a novel tree-based binary classification algorithm that can control the number of false negatives with a mathematical guarantee, based on Neyman–Pearson (NP) Lemma. This classifier is evaluated on the data obtained from different heart studies and it predicts the risk of cardiac disease, not only with comparable accuracy and AUC-ROC score but also with full control over the number of false negatives. The methodology used to construct this classifier can be expanded to many more use cases, not only in medical disease diagnosis but also beyond as shown from analysis on different diverse datasets.

Device Free Detection in Impulse Radio Ultrawide Bandwidth Systems.

In this paper, an analytical framework is presented for device detection in an impulse radio (IR) ultra-wide bandwidth (UWB) system and its performance analysis is carried out. The Neyman–Pearson (NP) criteria is employed for this device-free detection. Different from the frequency-based approaches, the proposed detection method utilizes time domain concepts. The characteristic function (CF) is utilized to measure the moments of the presence and absence of the device. Furthermore, this method is easily extendable to existing device-free and device-based techniques. This method can also be applied to different pulse-based UWB systems which use different modulation schemes compared to IR-UWB. In addition, the proposed method does not require training to measure or calibrate the system operating parameters. From the simulation results, it is observed that an optimal threshold can be chosen to improve the ROC for UWB system. It is shown that the probability of false alarm, , has an inverse relationship with the detection threshold and frame length. Particularly, to maintain for a frame length of 300 ns, it is required that the threshold should be greater than . It is also shown that for a fix , the probability of detection increases with an increase in interference-to-noise ratio (INR). Furthermore, approaches 1 for INR dB even for a very low i.e., . It is also shown that a 2 times increase in the interference energy results in a 3 dB improvement in INR for a fixed and . Finally, the derived performance expressions are corroborated through simulation.

An integrated scheme for dynamic security assessment considering misclassification constraint based on umbrella Neyman-Pearson classifiers

Power system is one of the most complex dynamic industrial systems with the continuous expansion of the system scale and extensive application of phasor measurement units (PMUs). Dynamic security assessment (DSA) misclassification of the system will bring unpredictable consequences. Therefore, an integrated scheme for DSA that considers misclassification constraint is proposed in this paper to satisfy practical applications. The scheme includes a feature selection process and a DSA model, which are based on the bagging nearest-neighbor prediction independence test (BNNPT) and umbrella Neyman-Pearson (NP) classifiers, respectively. The BNNPT can explore the relationships between variables and the dynamic security index (DSI) to screen out the crucial features highly related to the DSI. By controlling the Type I error threshold and the cyclic split training mode, umbrella NP classifiers can effectively constrain Type I error, reduce the effect of misclassification, and improve DSA for the secure operation of the system. The performance of this integrated scheme is demonstrated through tests on the IEEE 39-bus system and a practical 500-bus system.

Inhomogeneous Analysis of Novel Model of CFAR Approaches to Detect Two-Degrees of Freedom Partially-Correlated χ2-Targets

Reliable and high performance radar systems have ubiquitous demand. The operation of such systems is affected by the presence of natural and artificial noise sources. One of the basic radar concepts is to decide whether the target is present or not. Meanwhile, the general objective of all radar detection schemes is to ensure that false alarms don't fluctuate randomly. Thus, to cope with an inhomogeneous changing clutter environment, it is beneficial to be able to detect both high- and low-fidelity targets while maintaining the rate of false alarm fixed. This calls for an adaptive thresholding strategy that vary the detection threshold as a function of the sensed environment, and most modern radars implement this approach automatically. The feature of constant false alarm rate (CFAR) activates the threshold in such a way that it becomes adaptive to the local clutter environment. Many alternatives have been proposed to achieve such demanded property. Owing to the diversity of the radar search environment (target multiplicity & clutter edges), there exists no universal CFAR procedure. This prompts the necessity to investigate the composite architecture as a novel strategy. The goal of this paper is to analyze the fusion of CA, OS, and TM processors in post-detection integration of M-pulses. The primary and outlying targets are assumed to obey χ 2 -distribution with two-degrees of freedom in their fluctuation. Closed-form expression is derived for the detection performance. Our simulation results show robust behavior of the new model in the absence as well as in the presence of outlying targets. In addition, a significant improvement of the detection performance of novel strategy over the individual CFAR detectors is noticed. Moreover, the outweighing, over Neyman-Pearson (N-P) detector, of the fusion model, in ideal background, is evidently demonstrated. This ability to obtain improved performance compared to existing models is the major contribution of this work.

Mutual interference alignment for co-existing radar and communication systems

In this paper, we propose a mutual interference alignment (IA) method for interference elimination in co-existing multiple-input multiple-output (MIMO) radar and multi-user MIMO communication systems, namely, radar-communication systems. Traditional IA methods proposed for communication systems cannot be directly adopted in the radar-communications due to failure to fulfill the requirements of radar target detection. In contrast, our proposed mutual IA method can efficiently improve the signal-to-noise ratio (SNR) needed by radar functionalities without reducing the degrees-of-freedom (DoFs) of communication system, which implies no degradation for the communication performance. The algorithm idea is first to align all interferences at every receiver to one signal space, then the interferences can be easily canceled by receive filter. It is examined by the Neyman-Pearson (NP) test for radar functionality, while the communication performance is evaluated by DoFs and bit error rate (BER) analysis. Both theory analysis and numerical results are provided to validate the effectiveness of the proposed mutual IA method for co-existing radar and communication systems.

Extended GLRT Detection of Moving Targets for Multichannel SAR Based on Generalized Steering Vector.

A generalized likelihood ratio test (GLRT) with the constant false alarm rate (CFAR) property was recently developed for adaptive detection of moving targets in focusing synthetic aperture radar (SAR) images. However, in the multichannel SAR-ground moving-target indication (SAR-GMTI) system, image defocus is inevitable, which will remarkably degrade the performance of the GLRT detector, especially for the lower radar cross-section (RCS) and slower radial velocity moving targets. To address this issue, based on the generalized steering vector (GSV), an extended GLRT detector is proposed and its performance is evaluated by the optimum likelihood ratio test (LRT) in the Neyman-Pearson (NP) criterion. The joint data vector formulated by the current cell and its adjacent cells is used to obtain the GSV, and then the extended GLRT is derived, which coherently integrates signal and accomplishes moving-target detection and parameter estimation. Theoretical analysis and simulated SAR data demonstrate the effectiveness and robustness of the proposed detector in the defocusing SAR images.

Stochastic Resonance Effect in Optimal Decision Solution Under Neyman–Pearson Criterion

In this paper, stochastic resonance effect on the optimal detection under Neyman–Pearson (NP) criterion is investigated for a general nonlinear system. To this end, a noise enhanced detection optimization problem for maximizing the probability of detection under a constant constraint on the probability of false-alarm is formulated, where an additive noise is added to the nonlinear system input and the final decision is made based on the system output according to the NP criterion. Firstly, the noise modified NP decision rule is derived. Since one additive noise corresponds to one noise modified NP decision rule, the noise-modified NP decision rule can be viewed as a function of the additive noise. Then, the improvability of the optimal detection performance under NP criterion via the proposed noise-modified decision solution is simply discussed. The optimal additive noise is deduced as a random signal of no more than two constant vectors and the corresponding noise-modified NP decision rule is also determined. Finally, the performance comparisons between the original and the noise-modified optimal NP decision solutions for the sine transform system and the Amplitude limit system are made to illustrate the theoretical results.

Towards Real-Time Detection of Auditory Steady-State Responses: A Comparative Study

To objectively evaluate weak neural responses such as auditory steady-state responses (ASSRs) often involves repetitive measurements, after which the accumulated trials are averaged or analyzed by statistical methods to detect a significant response. Such detection methods are often performed off-line on all measured trials. However, the number of required trials, and therefore the measurement time, is often heuristically chosen, and might be suboptimal. In this work, we compared three real-time signal-detection algorithms that could yield improved detection performance, at reduced measurement time. The classical Neyman-Pearson (NP) detector was evaluated by quantifying the signal-to-noise ratio (SNR), and detection probability of ASSRs, as a function of accumulated trial number. We also analyzed the performance of the Bayes factor (BF) to detect ASSRs at different thresholds. Finally, we modified the sequential probability ratio test (SPRT), with a cropped maximum likelihood (ML) estimator, such that it can detect ASSRs (with unknown SNRs) sequentially. We compared the three real-time detectors by using Monte Carlo simulation, and evaluated their performance on detecting ASSRs, generated from the superposition of a pair of amplitude-modulated (AM) tones near 40 Hz in the EEG of nine subjects with normal hearing. The low-order ASSRs (i.e., envelope frequency-following responses) have been sufficiently evaluated in the literature. However, the higher-order ASSRs might reflect further nonlinear mechanisms in the upper ascending auditory pathway. Results show that the real-time detectors can detect not only all low-order ASSRs but also higher-order ASSRs at frequencies with lower SNR. The NP detector yielded the best simulation and actual detection performance. For all subjects, the second-order ASSRs could already be detected with the NP and BF detectors within five trials, but more trials were needed for the modified SPRT detector. In general, higher-order ASSRs require more trials to detect, with the exception of ASSRs near 40 Hz and 80 Hz. In conclusion, compared with traditional off-line detectors, both (real-time) NP and BF detectors showed improved detection performance, and their application in EEG experiments can save valuable measurement time.