Sequential Detection Procedure Research Articles

Sequential label shift detection in classification data: An application to dengue fever.

Classifiers have been developed to help diagnose dengue fever in patients presenting with febrile symptoms. However, classifier predictions often rely on the assumption that new observations come from the same distribution as training data. If the population prevalence of dengue changes, as would happen with a dengue outbreak, it is important to raise an alarm as soon as possible, so that appropriate public health measures can be taken and also so that the classifier can be re-calibrated. In this paper, we consider the problem of detecting such a change in distribution in sequentially-observed, unlabeled classification data. We focus on label shift changes to the distribution, where the class priors shift but the class conditional distributions remain unchanged. We reduce this problem to the problem of detecting a change in the one-dimensional classifier scores, leading to simple nonparametric sequential changepoint detection procedures. Our procedures leverage classifier training data to estimate the detection statistic, and converge to their parametric counterparts in the size of the training data. In simulated outbreaks with real dengue data, we show that our method outperforms other detection procedures in this label shift setting.

Truncated sequential change-point detection for Markov chains with applications in the epidemic statistical analysis

In this study, we consider disorder detection problems for statistical models with dependent observations defined by Markov chains in a Bayesian setting for a uniform prior distribution when the number of observations is limited by some known quantity. To achieve this, a new optimal sequential detection procedure is constructed using optimal stopping methods. This procedure minimizes the average delay time in the class of sequential procedures with false alarm probabilities that do not exceed a fixed value. The main difference between the proposed detection procedure and the conventional ones is that it is based not on the posterior probabilities but on the weighted Shiryaev-Roberts statistic. This allows for optimal detection in a nonasymptotic sense over any bounded time interval. Thereafter, we applied the constructed procedures to the early detection problem for the beginning of the epidemic spread. We used two epidemic models: the binomial model proposed by Baron, Choudhary, and Yu (2013) and the model based on the Gaussian approximation introduced by Pergamenchtchikov, Tartakovsky, and Spivak (2022). The theoretical results were confirmed through numerical simulations using the Monte Carlo method.

Selecting a significance level in sequential testing procedures for community detection

While there have been numerous sequential algorithms developed to estimate community structure in networks, there is little available guidance and study of what significance level or stopping parameter to use in these sequential testing procedures. Most algorithms rely on prespecifiying the number of communities or use an arbitrary stopping rule. We provide a principled approach to selecting a nominal significance level for sequential community detection procedures by controlling the tolerance ratio, defined as the ratio of underfitting and overfitting probability of estimating the number of clusters in fitting a network. We introduce an algorithm for specifying this significance level from a user-specified tolerance ratio, and demonstrate its utility with a sequential modularity maximization approach in a stochastic block model framework. We evaluate the performance of the proposed algorithm through extensive simulations and demonstrate its utility in controlling the tolerance ratio in single-cell RNA sequencing clustering by cell type and by clustering a congressional voting network.

Abstract 2784: Automated detection of IFN-γ, ERBB2, IgG-Kappa mRNA and protein in FFPE tissues

Abstract Introduction: In recent years, the detection of RNA molecules has become increasingly important for the analysis of the physiological status of cells within tissues. This has complemented the established immunohistochemistry (IHC)/immunofluorescent (IF) technologies for detection of protein targets on cells and tissues. Combined detection of RNA and protein targets is more informative and is providing better insights into the physiological status of cells and tissues than the individual tests alone. However, the sequential detection procedures applied in IHC/IF and RNA in situ hybridization (RISH) can be tricky when performed manually. Thus, continuous development of automation platforms is geared toward simplifying the application of these technologies. In this study, we report the automated co-detection of ERBB2, IgG-Kappa and INF Gamma (IFN-γ) protein and mRNA on the ONCORE Pro platform. Materials & Methods: Tissues used in this study were FFPE and sectioned at 5 microns. Slides were deparaffinized, dehydrated and peroxidase blocked before performing HIER or enzymatic digestion with pronase. Then, for ERBB2 and IG-Kappa, the mRNA detection was followed by IF detection of the protein target. For IFN-γ, this order was reversed. The Advanced Cell Diagnostics (ACD) recommended protocol for detection of RNA was modified for fluorescent detection and adapted for application on the ONCORE Pro. mRNA was visualized using Opal 520 with green fluorescence and protein targets with mouse or goat antibody-CF543 orange fluorescence. Antibodies and reagents were provided by Biocare Medical and Abcam, in situ probes and detection by ACD, and Opal 520 by Akoya Biosciences. Results: In our experiments, for the detection of mRNA of ERBB2 and IG-Kappa, we used pronase, and this seemed not to have any effect on the further detection of their protein targets. However, for IFN-γ, we observed that pronase seemed to destroy the target protein, and when the detection order was reversed (i.e. the protein first and then the mRNA), both targets could be seen under the microscope. The sequential approach worked well with generating a clear and consistent staining of 3+ on a staining strength scale of 1-3 for IF and ISH, while the negative controls displayed no specific staining and were clean. Detection of IFN-γ protein was not trivial, we observed certain tissue-specific patterns that were identified when detecting IFN by IHC versus the mRNA in certain tissues indicating the possibility of a non-perfect correlation. Indeed, co-application of RISH simplified the identification of the true positive cells and target areas on tissues. Conclusions: Application of the combined ISH-IF detection approach is beneficial. When added this to the apparent advantages of full automation (ONCORE Pro), this blend makes these technologies more attractive for studying the physiological status of immuno-oncology markers in target tissues and organs. Citation Format: Julio S. Masabanda, Sara Figueroa, Joseph Vargas, Jason Ramos. Automated detection of IFN-γ, ERBB2, IgG-Kappa mRNA and protein in FFPE tissues [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 2784.

Nonparametric sequential change-point detection for multivariate time series based on empirical distribution functions

The aim of sequential change-point detection is to issue an alarm when it is thought that certain probabilistic properties of the monitored observations have changed. This work is concerned with nonparametric, closed-end testing procedures based on differences of empirical distribution functions that are designed to be particularly sensitive to changes in the contemporary distribution of multivariate time series. The proposed detectors are adaptations of statistics used in a posteriori (offline) change-point testing and involve a weighting allowing to give more importance to recent observations. The resulting sequential change-point detection procedures are carried out by comparing the detectors to threshold functions estimated through resampling such that the probability of false alarm remains approximately constant over the monitoring period. A generic result on the asymptotic validity of such a way of estimating a threshold function is stated. As a corollary, the asymptotic validity of the studied sequential tests based on empirical distribution functions is proven when these are carried out using a dependent multiplier bootstrap for multivariate time series. Large-scale Monte Carlo experiments demonstrate the good finite-sample properties of the resulting procedures. The application of the derived sequential tests is illustrated on financial data.

Sequential change-point detection in a multinomial logistic regression model

Abstract Change-point detection in categorical time series has recently gained attention as statistical models incorporating change-points are common in practice, especially in the area of biomedicine. In this article, we propose a sequential change-point detection procedure based on the partial likelihood score process for the detection of changes in the coefficients of multinomial logistic regression model. The asymptotic results are presented under both the null of no change and the alternative of changes in coefficients. We carry out a Monte Carlo experiment to evaluate the empirical size of the proposed procedure as well as its average run length. We illustrate the method by using data on a DNA sequence. Monte Carlo experiments and real data analysis demonstrate the effectiveness of the proposed procedure.

Detection of Outliers in Projection-Based Modeling.

Previously, we have introduced an approach for calculation of the full object distance in the frame of Principal Component Analysis that can be applied to data exploration and classification. Now, a similar approach has been developed for regression problems in which a total distance can be calculated for every sample in projection modeling. Based on the total distance, a threshold for outlier detection has been developed by means of a data-driven estimation of the degrees of freedom and scaling parameters for the partial distances in the projection models. A joint threshold is used as a basis for a sequential outlier detection procedure. The iterative nature of the procedure helps to overcome masking effect in outliers, and a backward step eliminates swamping effects. Two real examples are used for illustration. The first dataset represents capsules filled with specially prepared mixtures of an active pharmaceutical ingredient and a number of excipients. This dataset is used to illustrate the behavior of possible outliers in the regression model and their corresponding locations in the X- and XY-distance plots. The second dataset consists of spectra of 135 whole wheat samples used for the prediction of protein, gluten, and moisture content. This dataset is used for a demonstration of the step-by-step application of the sequential procedure for outlier detection.

Sketching for sequential change-point detection

We present sequential change-point detection procedures based on linear sketches of high-dimensional signal vectors using generalized likelihood ratio (GLR) statistics. The GLR statistics allow for an unknown post-change mean that represents an anomaly or novelty. We consider both fixed and time-varying projections, derive theoretical approximations to two fundamental performance metrics: the average run length (ARL) and the expected detection delay (EDD); these approximations are shown to be highly accurate by numerical simulations. We further characterize the relative performance measure of the sketching procedure compared to that without sketching and show that there can be little performance loss when the signal strength is sufficiently large, and enough number of sketches are used. Finally, we demonstrate the good performance of sketching procedures using simulation and real-data examples on solar flare detection and failure detection in power networks.

Quickest Change Detection in the Presence of a Nuisance Change

In the quickest change detection problem in which both nuisance and critical changes may occur, the objective is to detect the critical change as quickly as possible without raising an alarm when either there is no change or a nuisance change has occurred. A window-limited sequential change detection procedure based on the generalized likelihood ratio test statistic is proposed. A recursive update scheme for the proposed test statistic is developed and is shown to be asymptotically optimal under mild technical conditions. In the scenario where the post-change distribution belongs to a parametrized family, a generalized stopping time and a lower bound on its average run length are derived. The proposed stopping rule is compared with the FMA stopping time and the naive 2-stage procedure that detects the nuisance or critical change using separate CuSum stopping procedures for the nuisance and critical changes. Simulations demonstrate that the proposed rule outperforms the FMA stopping time and the 2-stage procedure, and experiments on a real dataset on bearing failure verify the performance of the proposed stopping time.

A Binning Approach to Quickest Change Detection With Unknown Post-Change Distribution

The problem of quickest detection of a change in distribution is considered under the assumption that the prechange distribution is known, and the postchange distribution is only known to belong to a family of distributions distinguishable from a discretized version of the prechange distribution. A sequential change detection procedure is proposed that partitions the sample space into a finite number of bins and monitors the number of samples falling into each of these bins to detect the change. A test statistic that approximates the generalized likelihood ratio test is developed. It is shown that the proposed test statistic can be efficiently computed using a recursive update scheme, and a procedure for choosing the number of bins in the scheme is provided. Various asymptotic properties of the test statistic are derived to offer insights into its performance tradeoff between average detection delay and average run length to false alarm. Testing on synthetic and real data demonstrates that our approach is comparable or better in the performance to existing nonparametric change detection methods.

Two-Step Sequential Detection in Agile-Beam Radars: Performance and Tradeoffs

In this paper, we consider a surveillance radar equipped with an electronically scanned antenna and study the performance of a two-step sequential detection procedure, where, for each resolution cell, a second observation is taken if a reliable decision cannot be made after the first one. At the design stage, we optimize the available degrees of freedom (namely, dwell time and detection thresholds) so as to maximize the detection rate (DR), defined as the average number of detections from a target per unit of time, under a constraint on the false alarm rate (FAR), which is the average number of false alarms per unit of time from the inspected area. This is motived by the fact that DR and FAR, beside of being per se meaningful figures of merit for parameter tuning, allow a fair performance comparison among detection strategies with different scanning policies. Examples are presented to illustrate the effects of this design philosophy under the two relevant situations of a slowly fluctuating and a fast-fluctuating target response, also in the presence of clutter.

Multichannel Sequential Detection—Part I: Non-i.i.d. Data

We consider the problem of sequential signal detection in a multichannel system assuming that the number and location of signals are either unknown or only partially known a priori . We focus on the design and analysis of two sequential hypothesis tests: the generalized sequential likelihood ratio test and the mixture sequential likelihood ratio test. We develop an asymptotic theory for a general stochastic model, where the various data streams can be coupled and correlated, and the data in each stream can be dependent and non-identically distributed. Specifically, we show that the two proposed sequential detection procedures asymptotically minimize the expected sample size and even higher moments of the sample size in the class of hypothesis tests with given probabilities of errors under weak distributional assumptions. We also propose efficient importance sampling algorithms for estimating error probabilities of the sequential tests by Monte Carlo simulation. The general theory is illustrated with several practical examples, such as the detection of signals, in Gaussian hidden Markov models, white Gaussian noises with unknown intensity, and testing of the first-order autoregression’s correlation coefficient. Finally, we illustrate our asymptotic results and compare the two proposed procedures with a simulation study.

Asymptotically optimal pointwise and minimax quickest change-point detection for dependent data

We consider the quickest change-point detection problem in pointwise and minimax settings for general dependent data models. Two new classes of sequential detection procedures associated with the maximal “local” probability of a false alarm within a period of some fixed length are introduced. For these classes of detection procedures, we consider two popular risks: the expected positive part of the delay to detection and the conditional delay to detection. Under very general conditions for the observations, we show that the popular Shiryaev–Roberts procedure is asymptotically optimal, as the local probability of false alarm goes to zero, with respect to both these risks pointwise (uniformly for every possible point of change) and in the minimax sense (with respect to maximal over point of change expected detection delays). The conditions are formulated in terms of the rate of convergence in the strong law of large numbers for the log-likelihood ratios between the “change” and “no-change” hypotheses, specifically as a uniform complete convergence of the normalized log-likelihood ratio to a positive and finite number. We also develop tools and a set of sufficient conditions for verification of the uniform complete convergence for a large class of Markov processes. These tools are based on concentration inequalities for functions of Markov processes and the Meyn–Tweedie geometric ergodic theory. Finally, we check these sufficient conditions for a number of challenging examples (time series) frequently arising in applications, such as autoregression, autoregressive GARCH, etc.

Coupled sequential detection and estimation procedure for track‐before‐detect

In this study, the authors consider the coupled sequential detection and estimation of weak targets for track-before-detect algorithm in long range early warning radar. They present a new strategy and develop a coupled cost function consisting of detection, position estimation and tracking estimation loss. In the proposed procedure, the sequential stopping and detection rules are simplified. Moreover, they derive the lower bounds of the detection and estimation probabilities, which can be used to design the suitable related parameter values. Finally, several numerical simulations of the new strategy and the comparisons are presented and discussed.

Hybrid algorithms for multiple change-point detection in biological sequences.

Array comparative genomic hybridization (aCGH) is one of the techniques that can be used to detect copy number variations in DNA sequences in high resolution. It has been identified that abrupt changes in the human genome play a vital role in the progression and development of many complex diseases. In this study we propose two distinct hybrid algorithms that combine efficient sequential change-point detection procedures (the Shiryaev-Roberts procedure and the cumulative sum control chart (CUSUM) procedure) with the Cross-Entropy method, which is an evolutionary stochastic optimization technique to estimate both the number of change-points and their corresponding locations in aCGH data. The proposed hybrid algorithms are applied to both artificially generated data and real aCGH experimental data to illustrate their usefulness. Our results show that the proposed methodologies are effective in detecting multiple change-points in biological sequences of continuous measurements.

Bootstrapping sequential change-point tests for linear regression

Bootstrap methods for sequential change-point detection procedures in linear regression models are proposed. The corresponding monitoring procedures are designed to control the overall significance level. The bootstrap critical values are updated constantly by including new observations obtained from the monitoring. The theoretical properties of these sequential bootstrap procedures are investigated, showing their asymptotic validity. Bootstrap and asymptotic methods are compared in a simulation study, showing that the studentized bootstrap tests hold the overall level better especially for small historic sample sizes while having a comparable power and run length.

Sequential algorithm for life threatening cardiac pathologies detection based on mean signal strength and EMD functions

BackgroundVentricular tachycardia (VT) and ventricular fibrillation (VF) are the most serious cardiac arrhythmias that require quick and accurate detection to save lives. Automated external defibrillators (AEDs) have been developed to recognize these severe cardiac arrhythmias using complex algorithms inside it and determine if an electric shock should in fact be delivered to reset the cardiac rhythm and restore spontaneous circulation. Improving AED safety and efficacy by devising new algorithms which can more accurately distinguish shockable from non-shockable rhythms is a requirement of the present-day because of their uses in public places.MethodIn this paper, we propose a sequential detection algorithm to separate these severe cardiac pathologies from other arrhythmias based on the mean absolute value of the signal, certain low-order intrinsic mode functions (IMFs) of the Empirical Mode Decomposition (EMD) analysis of the signal and a heart rate determination technique. First, we propose a direct waveform quantification based approach to separate VT plus VF from other arrhythmias. The quantification of the electrocardiographic waveforms is made by calculating the mean absolute value of the signal, called the mean signal strength. Then we use the IMFs, which have higher degree of similarity with the VF in comparison to VT, to separate VF from VTVF signals. At the last stage, a simple rate determination technique is used to calculate the heart rate of VT signals and the amplitude of the VF signals is measured to separate the coarse VF from VF. After these three stages of sequential detection procedure, we recognize the two components of shockable rhythms separately.ResultsThe efficacy of the proposed algorithm has been verified and compared with other existing algorithms, e.g., HILB [1], PSR [2], SPEC [3], TCI [4], Count [5], using the MIT-BIH Arrhythmia Database, Creighton University Ventricular Tachyarrhythmia Database and MIT-BIH Malignant Ventricular Arrhythmia Database. Four quality parameters (e.g., sensitivity, specificity, positive predictivity, and accuracy) were calculated to ascertain the quality of the proposed and other comparing algorithms. Comparative results have been presented on the identification of VTVF, VF and shockable rhythms (VF + VT above 180 bpm).ConclusionsThe results show significantly improved performance of the proposed EMD-based novel method as compared to other reported techniques in detecting the life threatening cardiac arrhythmias from a set of large databases.

Sampling Schemes for Sequential Detection With Dependent Observations

Several sampling schemes and their corresponding sequential detection procedures in autoregressive noise are presented in this paper. Two of them use uniform sampling procedures with high and low sampling rates, respectively. The other two employ groups of samples, which are separated by long intergroup delays such that the intergroup correlations are negligible. One of the group-sampling schemes also employs optimal signaling waveforms to further improve its energy-efficiency. In all the schemes, data sampling and transformation are designed in such a way that Wald's sequential probability ratio test (SPRT) can still be implemented. The performances of different schemes, in terms of average termination time (ATT), are derived analytically. When all the schemes employ the same sampling interval and under a constant signal amplitude constraint, their performances are compared through analytical and numerical methods. In addition, under a constant power constraint, their ATTs and energy-efficiency are compared. It is theoretically proved that the scheme using groups of samples with the optimal signaling waveform is the most energy-efficient.

Stochastic Resonance in Sequential Detectors

Stochastic resonance (SR) is a nonlinear phenomenon known in physics that has attracted recent interest in the signal-processing literature, and specifically in the context of detection. We investigate the SR effect arising in sequential detectors for shift-in-mean binary hypothesis testing and characterize the optimal resonance as the solution of specific optimization problems. One particular (and at first glance perhaps counterintuitive) finding is that certain sequential detection procedures can be made more efficient by randomly adding or subtracting a suitable constant value to the data at the input of the detector.

Sequential Change-Point Detection Procedures That are Nearly Optimal and Computationally Simple

Sequential schemes for detecting a change in distribution often require that all of the observations be stored in memory. Lai (1995, Journal of Royal Statistical Society, Series B 57: 613–658) proposed a class of detection schemes that enable one to retain a finite window of the most recent observations, yet promise first-order optimality. The asymptotics are such that the window size is asymptotically unbounded. We argue that what's of computational importance isn't having a finite window of observations, but rather making do with a finite number of registers. We illustrate in the context of detecting a change in the parameter of an exponential family that one can achieve eventually even second-order asymptotic optimality through using only three registers for storing information of the past. We propose a very simple procedure, and show by simulation that it is highly efficient for typical applications.