Sequential Probability Ratio Test Research Articles

Bayesian safety surveillance with adaptive bias correction.

Postmarket safety surveillance is an integral part of mass vaccination programs. Typically relying on sequential analysis of real-world health data as they accrue, safety surveillance is challenged by sequential multiple testing and by biases induced by residual confounding in observational data. The current standard approach based on the maximized sequential probability ratio test (MaxSPRT) fails to satisfactorily address these practical challenges and it remains a rigid framework that requires prespecification of the surveillance schedule. We develop an alternative Bayesian surveillance procedure that addresses both aforementioned challenges using a more flexible framework. To mitigate bias, we jointly analyze a large set of negative control outcomes that are adverse events with no known association with the vaccines in order to inform an empirical bias distribution, which we then incorporate into estimating the effect of vaccine exposure on the adverse event of interest through a Bayesian hierarchical model. To address multiple testing and improve on flexibility, at each analysis timepoint, we update a posterior probability in favor of the alternative hypothesis that vaccination induces higher risks of adverse events, and then use it for sequential detection of safety signals. Through an empirical evaluation using six US observational healthcare databases covering more than 360 million patients, we benchmark the proposed procedure against MaxSPRT on testing errors and estimation accuracy, under two epidemiological designs, the historical comparator and the self-controlled case series. We demonstrate that our procedure substantially reduces Type 1 error rates, maintains high statistical power and fast signal detection, and provides considerably more accurate estimation than MaxSPRT. Given the extensiveness of the empirical study which yields more than 7 million sets of results, we present all results in a public R ShinyApp. As an effort to promote open science, we provide full implementation of our method in the open-source R package EvidenceSynthesis.

Assessment of the learning curve of peribulbar blocks using the Learning-Curve Cumulative Sum Method (LC-CUSUM): an observational study

IntroductionThis study aimed to assess the learning curves of peribulbar anesthesia and estimate the number of blocks needed to attain proficiency. MethodsAnonymized records of sequential peribulbar blocks performed by first-year anesthesia residents were analyzed. The block sequential number and the outcomes were extracted from each record. Success was defined as a complete sensory and motor block of the eye, and failure was defined as an incomplete block requiring supplemental local anesthetic injections or general anesthesia. Learning curves using the LC-CUSUM method were constructed, aiming for acceptable and unacceptable failure rates of 10% and 20%, and 10% probability of type I and II errors. Simulations were used to determine the proficiency limit h0. Residents whose curves reached h0 were considered proficient. The Sequential Probability Ratio Test Cumulative Sum Method (SPRT-CUSUM) was used for follow-up. ResultsThirty-nine residents performed 2076 blocks (median = 52 blocks per resident; Interquartile Range (IQR) [range] = 27–78 [4–132]). Thirty residents (77%) achieved proficiency after a median of 13 blocks (13–24 [13–24]). ConclusionsThe LC-CUSUM is a robust method for detecting resident proficiency at peribulbar anesthesia, defined as success rates exceeding 90%. Accordingly, 13 to 24 supervised double-injection peribulbar blocks are needed to attain competence at peribulbar anesthesia.

A truncated test scheme design method for success-failure in-orbit tests

Based on the success-failure test feature of in-orbit tests (IOTs) for typical space equipment, this paper presents a method for designing a truncated test scheme for success-failure in-orbit tests. With this method, a small upper boundary of the sample size for the IOT verification test can be obtained before the test starts. The method introduces the truncated Bayes-sequential mesh test (SMT) method into the design of the IOT verification test scheme and greatly compresses the continuous test area by incorporating optimization theory, resulting in a smaller upper limit of the IOT sample size. First, this paper derives a specific calculation formula for the Bayes-SMT critical line. Second, the Markov chain model is adopted to calculate the occurrence probabilities of each acceptance and rejection point through state transition. Finally, an optimal truncated test optimization algorithm based on the augmented lagrangian genetic algorithm is proposed. Simulation tests show that, compared with the classical single sampling method, the truncated sequential probability ratio test method, the truncated SMT method, and the truncated Bayes-SMT method based on step-by-step calculation, the method presented in this paper can be used to obtain a sequential test scheme with smaller truncated sample size.

Location Privacy-Aware and Energy-Efficient Offloading for Distributed Edge Computing

Driven by the ever-increasing scale and intensity of the computing tasks arising from various mobile applications, distributed edge computing has fostered wide research interests. It can effectively reduce the task processing delay by partitioning the original large-scale task into several small subtasks and offloading them to multiple edge nodes (ENs) for parallel computing. In edge computing, as the mobile user usually tends to offload computing tasks to closer ENs to save transmit power, the attacker may stealthily infer user location by exploiting this feature. Although there have been some pioneering works on offloading related location privacy, they mainly focused on the scenario where each task can only be offloaded to a single EN, and may not be directly applicable to distributed edge computing. Besides, the privacy issues considered in existing works are mainly based on good heuristics, and there is still a lack of concrete examples of location privacy attacks in edge computing. To the best of our knowledge, the location privacy issue in distributed edge computing still remains largely unexplored in existing literature. With this consideration, a location inference attack based on matrix sequential probability ratio test (MSPRT) is identified in this work. Besides, a countermeasure based on dynamic multi-EN selection is proposed, together with a location privacy-aware and energy-efficient distributed offloading scheme based on the generic Lyapunov optimization framework. Both theoretic analysis and simulations based on real-world channel measurements are employed to validate the feasibility of the identified MPSRT attack and the effectiveness of the proposed defense scheme.

Statistical rules for safety monitoring in clinical trials.

Protecting patient safety is an essential component of the conduct of clinical trials. Rigorous safety monitoring schemes are implemented for these studies to guard against excess toxicity risk from study therapies. They often include protocol-specified stopping rules dictating that an excessive number of safety events will trigger a halt of the study. Statistical methods are useful for constructing rules that protect patients from exposure to excessive toxicity while also maintaining the chance of a false safety signal at a low level. Several statistical techniques have been proposed for this purpose, but the current literature lacks a rigorous comparison to determine which method may be best suitable for a given trial design. The aims of this article are (1) to describe a general framework for repeated monitoring of safety events in clinical trials; (2) to survey common statistical techniques for creating safety stopping criteria; and (3) to provide investigators with a software tool for constructing and assessing these stopping rules. The properties and operating characteristics of stopping rules produced by Pocock and O'Brien-Fleming tests, Bayesian Beta-Binomial models, and sequential probability ratio tests (SPRTs) are studied and compared for common scenarios that may arise in phase II and III trials. We developed the R package "stoppingrule" for constructing and evaluating stopping rules from these methods. Its usage is demonstrated through a redesign of a stopping rule for BMT CTN 0601 (registered at Clinicaltrials.gov as NCT00745420), a phase II, single-arm clinical trial that evaluated outcomes in pediatric sickle cell disease patients treated by bone marrow transplant. Methods with aggressive stopping criteria early in the trial, such as the Pocock test and Bayesian Beta-Binomial models with weak priors, have permissive stopping criteria at late stages. This results in a trade-off where rules with aggressive early monitoring generally will have a smaller number of expected toxicities but also lower power than rules with more conservative early stopping, such as the O-Brien-Fleming test and Beta-Binomial models with strong priors. The modified SPRT method is sensitive to the choice of alternative toxicity rate. The maximized SPRT generally has a higher number of expected toxicities and/or worse power than other methods. Because the goal is to minimize the number of patients exposed to and experiencing toxicities from an unsafe therapy, we recommend using the Pocock or Beta-Binomial, weak prior methods for constructing safety stopping rules. At the design stage, the operating characteristics of candidate rules should be evaluated under various possible toxicity rates in order to guide the choice of rule(s) for a given trial; our R package facilitates this evaluation.

Fault-Tolerant SINS/Doppler Radar/Odometer Integrated Navigation Method Based on Two-Stage Fault Detection Structure.

To improve the reliability of strapdown inertial navigation system (SINS)/Doppler radar/odometer integrated navigation system, the federated Kalman filter with two-stage fault detection structure is designed, and a fault-tolerant SINS/Doppler radar/odometer integrated navigation method is proposed. Firstly, the pre-fault detection module sets before the local filter, and the residual chi-square test in the carrier coordinate system is selected to detect the abrupt faults of Doppler radar and odometer. Then, the secondary-fault detection module emplaces between the local filter and the main filter, and the sequential probability ratio test (SPRT) is selected to further detect the ramp faults that are difficult to detect by the residual chi-square test. To address the limitation of the SPRT in accurately determining the end time of faults, an improved SPRT is proposed. The improved SPRT reduces the influence of historical fault on the fault statistics by introducing forgetting factors to improve its sensitivity to the fault end. The simulation experiment indicates that the proposed method can quickly detect and isolate abrupt and ramp faults, and promptly restore normal operation of the integrated navigation system after the fault ends, effectively improving the fault tolerance and reliability of the integrated navigation system.

The two‐sided SPRT sign charts

AbstractA two‐sided nonparametric sequential probability ratio test (SPRT) control chart for process location is proposed. The mechanism of this chart is based on the SPRT for location based on the signs of the sequential observations that is extended for a two‐sided alternative hypothesis. Approximate expressions that relate the statistical performance measures of this chart with that of its corresponding one‐sided charts are obtained through a simulation study and are shown to be quite accurate. Furthermore, an approach to design the proposed chart is provided. The statistical performance of the proposed chart is numerically compared to that of the two‐sided Shewhart, cumulative sum, and exponentially weighted moving average sign charts for location and it is observed that in general the proposed chart significantly outperforms these charts for detecting a location shift of any size. An example is provided to illustrate the mechanism of the proposed chart. Also, the chart is extended for monitoring process dispersion.

Optimal designs of the omnibus SPRT control chart for joint monitoring of process mean and dispersion

The vast majority of control schemes related to the sequential probability ratio test (SPRT) are designed for the purpose of monitoring only the process mean. Nonetheless, most manufacturing processes are vulnerable to external factors that cause the process mean and variability to change simultaneously. It is, therefore, crucial to consider a joint scheme for monitoring both the location and scale parameters of a production process. In this article, we develop a scheme that combines both mean and variance information in a single SPRT, known as the omnibus SPRT (OSPRT) chart. Expressions for the run-length properties of the OSPRT chart are derived by means of the Markov chain approach. We also propose optimal designs for the OSPRT chart based on two different metrics, i.e. by minimising the average time to signal and the average extra quadratic loss. Through a comprehensive analysis, this article reveals that the optimal OSPRT chart outperforms the classical -S, weighted-loss cumulative sum, absolute-value SPRT, and two maximum weighted-moving-average-type charts. The optimal OSPRT chart also has the advantage of collecting a small number of samples on average before producing a decision. Finally, the implementation of the OSPRT chart is presented with a wire bonding industrial dataset.

Discriminating among Generalized Exponential, Weighted Exponential and Weibull Distributions

In this paper, we consider the problem of discriminating among three different positively skewed lifetime distributions, namely the generalized exponential distribution, the weighted exponential distribution, and the Weibull distribution. All of these distributions have been used quite effectively to analyze positively skewed lifetime data. We use the methods of the ratio of maximized likelihood, the minimum Kolmogorov distance, and the sequential probability ratio test to discriminate among these three distributions. The probability of correct selection is considered for each hypothesis based on several scenarios with Monte Carlo simulation. Real data applications are studied to illustrate the effectiveness of these proposed methods.

Study of a New Software Reliability Growth Model under Uncertain Operating Environments and Dependent Failures

The coronavirus disease (COVID-19) outbreak has prompted various industries to embark on digital transformation efforts, with software playing a critical role. Ensuring the reliability of software is of the utmost importance given its widespread use across multiple industries. For example, software has extensive applications in areas such as transportation, aviation, and military systems, where reliability problems can result in personal injuries and significant financial losses. Numerous studies have focused on software reliability. In particular, the software reliability growth model has served as a prominent tool for measuring software reliability. Previous studies have often assumed that the testing environment is representative of the operating environment and that software failures occur independently. However, the testing and operating environments can differ, and software failures can sometimes occur dependently. In this study, we propose a new model that assumes uncertain operating environments and dependent failures. In other words, the model proposed in this study takes into account a wider range of environments. The numerical examples in this study demonstrate that the goodness of fit of the new model is significantly better than that of the existing SRGM. Additionally, we show the utilization of the sequential probability ratio test (SPRT) based on the new model to assess the reliability of the dataset.

Unsupervised Anomaly Detection in Marine Diesel Engines using Transformer Neural Networks and Residual Analysis

This paper presents a novel unsupervised approach for detecting anomalies in marine diesel engines using a Transformer Neural Network based autoencoder (TAE) and residual analysis with Sequential Probability Ratio Test (SPRT) and Sum of Squares of Normalized Residuals (SSNR). The proposed method can capture temporal dependencies in normal timeseries data without the need for labeled failure data. To assess the effectiveness of the proposed approach, a dataset of faulty data is generated under the same operational profile as the normal training data. The model is trained using normal data, and the faulty data is reconstructed using the trained model. SPRT and SSNR are then used to analyze the residuals from the observed and reconstructed faulty data, with significant deviations exceeding a predefined threshold being identified as anomalous behavior. The experimental results demonstrate that the proposed approach can accurately and efficiently detect anomalies in marine diesel engines. Therefore, this approach can be considered as a promising solution for early anomaly detection, leading to timely maintenance and repair, and preventing costly downtime.

A Sequential Test of Traffic Intensity for the M/M/1 Queueing System

This paper deals with testing traffic intensity, which is the most important parameter of the queue system. The Wald-type sequential probability ratio test (SPRT) is performed to determine traffic intensity by taking advantage of the fact that the total number of customers arriving up to the kth service period, including the kth service period, has a negative binomial distribution. Acceptance and rejection limits obtained from the SPRT are arranged as a multiple sampling plan. This plan is applied with a finite Markov chain. Then, the Operating Characteristic (OC) function and the Average Sample Number (ASN) are obtained precisely. Obtaining the standard errors of the sample number compared to the studies on SPRT applications is the novelty brought by this paper. The sequential analysis method based on the finite Markov chain is also applied to a numerical sample.

A numerical approach to sequential multi-hypothesis testing for Bernoulli model

In this article, we deal with the problem of sequential testing of multiple hypotheses. The main goal is minimizing the expected sample size (ESS) under restrictions on the error probabilities. We take, as a criterion of minimization, a weighted sum of the ESSs evaluated at some points of interest in the parameter space, aiming at its minimization under restrictions on the error probabilities. We use a variant of the method of Lagrange multipliers based on the minimization of an auxiliary objective function (called Lagrangian) combining the objective function with the restrictions, taken with some constants called multipliers. Subsequently, the multipliers are used to make the solution comply with the restrictions. We develop a computer-oriented method of minimization of the Lagrangian function that provides, depending on the specific choice of the parameter points, optimal tests in different concrete settings, as in Bayesian, Kiefer-Weiss, and other settings. To exemplify the proposed methods for the particular case of sampling from a Bernoulli population, we develop a set of computer algorithms for designing sequential tests that minimize the Lagrangian function and for the numerical evaluation of test characteristics like the error probabilities and the ESS. We implement the algorithms in the R programming language. The program code is available in a public GitHub repository. For the Bernoulli model, we made a series of computer evaluations related to the optimality of sequential multi-hypothesis tests, in a particular case of three hypotheses. A numerical comparison with the matrix sequential probability ratio test is carried out. A method of solution of the multi-hypothesis Kiefer-Weiss problem is proposed and is applied for a particular case of three hypotheses in the Bernoulli model.

Optimal Information Usage in Binary Sequential Hypothesis Testing

An interesting question is whether an information theoretic interpretation can be given of optimal algorithms in sequential hypothesis testing. We prove that for the binary sequential probability ratio test of a continuous observation process, the mutual information between the observation process up to the decision time and the actual hypothesis conditioned on the decision variable is equal to zero. This result can be interpreted as an optimal usage of the information on the hypothesis available in the observations by the sequential probability ratio test. As a consequence, the mutual information between the random decision time of the sequential probability ratio test and the actual hypothesis conditioned on the decision variable is also equal to zero.

Quickest Inference of Network Cascades With Noisy Information

We study the problem of estimating the source of a network cascade given a time series of noisy information about the spread. Initially, there is a single vertex affected by the cascade (the source) and the cascade spreads in discrete time steps across the network. The cascade evolution is hidden, but one can observe a time series of noisy signals from each vertex. The time series of a vertex is assumed to be a sequence of i.i.d. samples from a pre-change distribution $Q_0$ before the cascade affects the vertex, and the time series is a sequence of i.i.d. samples from a post-change distribution $Q_1$ once the cascade has affected the vertex. Given the time series of noisy signals, which can be viewed as a noisy measurement of the cascade evolution, we aim to devise a procedure to reliably estimate the cascade source as fast as possible. We investigate Bayesian and minimax formulations of the source estimation problem, and derive near-optimal estimators for simple cascade dynamics and network topologies. In the Bayesian setting, an estimator which observes samples until the error of the Bayes-optimal estimator falls below a threshold achieves optimal performance. In the minimax setting, optimal performance is achieved by designing a novel multi-hypothesis sequential probability ratio test (MSPRT). We find that these optimal estimators require $\log \log n / \log (k - 1)$ observations of the noisy time series when the network topology is a $k$-regular tree, and $(\log n)^{\frac{1}{\ell + 1}}$ observations are required for $\ell$-dimensional lattices. Finally, we discuss how our methods may be extended to cascades on arbitrary graphs.

Is designed data collection still relevant in the big data era? – A discussion

Is designed data collection still relevant in the big data era? – A discussion

ALPHA: Audit that learns from previously hand-audited ballots

A risk-limiting election audit (RLA) offers a statistical guarantee: if the reported electoral outcome is incorrect, the audit has a known maximum chance (the risk limit) of not correcting it before it becomes final. BRAVO (Lindeman, Stark and Yates (In Proceedings of the 2011 Electronic Voting Technology Workshop/Workshop on Trustworthy Elections (EVT/WOTE’11) (2012) USENIX)), based on Wald’s sequential probability ratio test for the Bernoulli parameter, is the simplest and most widely tried method for RLAs, but it has limitations. It cannot accommodate sampling without replacement or stratified sampling which can improve efficiency and are sometimes required by law. It applies only to ballot-polling audits which are less efficient than comparison audits. It applies to plurality, majority, supermajority, proportional representation, and instant-runoff voting (IRV, using RAIRE (Blom, Stuckey and Teague (In Electronic Voting (2018) 17–34 Springer))) but not to other social choice functions for which there are RLA methods. And while BRAVO has the smallest expected sample size among sequentially valid ballot-polling-with-replacement methods when the reported vote shares are exactly correct, it can require arbitrarily large samples when the reported reported winner(s) really won but the reported vote shares are incorrect. ALPHA is a simple generalization of BRAVO that: (i) works for sampling with and without replacement, with and without weights, with and without stratification, and for Bernoulli sampling; (ii) works not only for ballot polling but also for ballot-level comparison, batch polling, and batch-level comparison audits; (iii) works for all social choice functions covered by SHANGRLA (Stark (In Financial Cryptography and Data Security (2020) Springer)), including approval voting, STAR-Voting, proportional representation schemes, such as D’Hondt and Hamilton, IRV, Borda count, and all scoring rules, and (iv) in situations where both ALPHA and BRAVO apply, requires smaller samples than BRAVO when the reported vote shares are wrong but the outcome is correct—five orders of magnitude in some examples. ALPHA includes the family of betting martingale tests in RiLACS (Waudby-Smith, Stark and Ramdas (In Electronic Voting. E-Vote-ID 2021 (2021) Springer)) with a different betting strategy parametrized as an estimator of the population mean and explicit flexibility to accommodate sampling weights and population bounds that change with each draw. A Python implementation is provided.

Non-invasive fetal genotyping for maternal alleles with droplet digital PCR: A comparative study of analytical approaches.

To develop a flexible droplet digital PCR (ddPCR) workflow to perform non-invasive prenatal diagnosis via relative mutation dosage (RMD) for maternal pathogenic variants with a range of inheritance patterns, and to compare the accuracy of multiple analytical approaches. Cell free DNA (cfDNA) was tested from 124 archived maternal plasma samples: 88 cases for sickle cell disease and 36 for rare Mendelian conditions. Three analytical methods were compared: sequential probability ratio testing (SPRT), Bayesian and z-score analyses. The SPRT, Bayesian and z-score analyses performed similarly well with correct prediction rates of 96%, 97% and 98%, respectively. However, there were high rates of inconclusive results for each cohort, particularly for z-score analysis which was 31% overall. Two samples were incorrectly classified by all three analytical methods; a false negative result predicted for a fetus affected with sickle cell disease and a false positive result predicting the presence of an X-linked IDS variant in an unaffected fetus. ddPCR can be applied to RMD for diverse conditions and inheritance patterns, but all methods carry a small risk of erroneous results. Further evaluation is required both to reduce the rate of inconclusive results and explore discordant results in more detail.

Self-Adaptive Air-path Health Management for a Heavy Duty-Diesel Engine

This paper presents the air-path health management strategy with the ability to estimate the mass-flows and mitigate (adapt to) the air-path faults in the exhaust system of a heavy-duty diesel combustion engine equipped with a twin-scroll turbine. Based on the engine component models applied in the quasi-steady-state mass-balancing approach, two main engine mass-flow quantities are estimated: the Air mass-flow (AMF) and the Exhaust gas recirculation (EGR) mass-flow. The health management system is monitoring for three kinds of air-path faults that can occur through the combustion engine operation, related either to the after-treatment system, EGR valve, or to the turbine balance valve hardware. For each fault, a fault-mitigation strategy based on in-observer-reconfigurable mass-balance equations with excluded faulty component model and utilized exhaust pressure sensor is proposed. The applied observer is using the iterated Kalman filter (IKF) as the core fault mitigating solver for the quasi-steady-state mass-balancing problem. It is further demonstrated how the individual faults are robustly isolated using the Sequential Probability Ratio Test (SPRT). The strategy and results are validated using the test cycle driving data.

ON PERFORMANCE OF ACCEPTANCE SAMPLING PLANS USING SEQUENTIAL PROBABILITY RATIO TEST BASED ON TRUNCATED LIFE TESTS USING SOME DISTRIBUTION FUNCTION

Sequential sampling procedure constitute a powerful tool to achieve experimental efficiency a sequence of sample are taken from the lot and allow the number of samples to be determined entirely by the result of the sampling process, it forms the basis of numerous sequential technique for different applications which was the special case for multiple acceptance plans, the performance of sequential probability ratio test provides better detection for a wide range of source strength and useful in choosing the best statistical test, which take less time to make decision, was developed on the software reliability measure based on failure intensity for the life time random variable for a truncated life test. In this paper, we considered the performance of acceptance sampling plans using sequential probability ration (SPRT) based on truncated life tests on Maxwell distribution and exponential distribution function in terms of obtaining the minimum number of sample sizes necessary to obtain specified average life time under a given consumer’s and producer’s risk. The SPRT provide an alternative to fix the plans that can help diminish producer and consumer risk of reaching at a wrong decision.