Multi-hypothesis Sequential Probability Ratio Test Research Articles

Quickest Inference of Network Cascades With Noisy Information

We study the problem of estimating the source of a network cascade given a time series of noisy information about the spread. Initially, there is a single vertex affected by the cascade (the source) and the cascade spreads in discrete time steps across the network. The cascade evolution is hidden, but one can observe a time series of noisy signals from each vertex. The time series of a vertex is assumed to be a sequence of i.i.d. samples from a pre-change distribution $Q_0$ before the cascade affects the vertex, and the time series is a sequence of i.i.d. samples from a post-change distribution $Q_1$ once the cascade has affected the vertex. Given the time series of noisy signals, which can be viewed as a noisy measurement of the cascade evolution, we aim to devise a procedure to reliably estimate the cascade source as fast as possible. We investigate Bayesian and minimax formulations of the source estimation problem, and derive near-optimal estimators for simple cascade dynamics and network topologies. In the Bayesian setting, an estimator which observes samples until the error of the Bayes-optimal estimator falls below a threshold achieves optimal performance. In the minimax setting, optimal performance is achieved by designing a novel multi-hypothesis sequential probability ratio test (MSPRT). We find that these optimal estimators require $\log \log n / \log (k - 1)$ observations of the noisy time series when the network topology is a $k$-regular tree, and $(\log n)^{\frac{1}{\ell + 1}}$ observations are required for $\ell$-dimensional lattices. Finally, we discuss how our methods may be extended to cascades on arbitrary graphs.

Performance Analysis of Sequential Detection of Primary User Number Based on Multihypothesis Sequential Probability Ratio Test

In cognitive radio networks, a priori information on the number of primary users (PUs) is helpful to estimate more specific parameters, such as carrier frequency, direction of arrival, and location. We propose a sequential canonical correlation technique (S-CCT) method to estimate the number of active PUs quickly and accurately. The proposed method is based on the classical CCT and multi-hypothesis sequential probability ratio test. We also deduce the asymptotic expressions of average sample number and detection probability. Simulation results show that our proposed S-CCT method can achieve better performance with fewer samples than the CCT.

Tracking-Aided Classification of Targets Using Multihypothesis Sequential Probability Ratio Test

This paper deals with target classification by using both feature data and kinematic measurements. The problem is tackled by multihypothesis sequential testing with embedded target tracking. We implement an Armitage sequential test for nonmaneuvering and maneuvering targets. Both (centralized and distributed) fusion architectures are used for the embedded tracking. The contributions of the kinematic measurements to classification are analyzed, and classification performance improvement is shown analytically for a special case. Numerical results are provided to demonstrate the performance of our algorithms.

Multi-alternative decision-making with non-stationary inputs.

One of the most widely implemented models for multi-alternative decision-making is the multihypothesis sequential probability ratio test (MSPRT). It is asymptotically optimal, straightforward to implement, and has found application in modelling biological decision-making. However, the MSPRT is limited in application to discrete (‘trial-based’), non-time-varying scenarios. By contrast, real world situations will be continuous and entail stimulus non-stationarity. In these circumstances, decision-making mechanisms (like the MSPRT) which work by accumulating evidence, must be able to discard outdated evidence which becomes progressively irrelevant. To address this issue, we introduce a new decision mechanism by augmenting the MSPRT with a rectangular integration window and a transparent decision boundary. This allows selection and de-selection of options as their evidence changes dynamically. Performance was enhanced by adapting the window size to problem difficulty. Further, we present an alternative windowing method which exponentially decays evidence and does not significantly degrade performance, while greatly reducing the memory resources necessary. The methods presented have proven successful at allowing for the MSPRT algorithm to function in a non-stationary environment.

Operating Characteristic and Average Sample Number of Binary and Multi-Hypothesis Sequential Probability Ratio Test

The operating characteristic (OC) and average sample number (ASN) of the sequential probability ratio test (SPRT) and multi-hypothesis SPRT (MSPRT) are studied. We consider the case where the observation sequence is independent but not necessarily identically distributed. Also, the thresholds for the test can be time varying. Based on the governing equations for OC and ASN of the SPRT developed in our previous work, a solution for the general case is proposed. The governing equations for OC and ASN of the MSPRT are also obtained. Numerical solutions for MSPRT are developed. Basically, the solutions rely on approximating the original test by truncation, that is, truncating the test at some finite time $K$ . We show that under some mild conditions, the approximation error diminishes as $K$ increases, at the cost of increased computation. Numerical examples are provided to demonstrate our solutions by comparing with Monte Carlo simulations, Simon’s lower bound, and Dragalin’s method (if available) for ASN.

Probabilistic Decision Making with Spikes: From ISI Distributions to Behaviour via Information Gain.

Computational theories of decision making in the brain usually assume that sensory 'evidence' is accumulated supporting a number of hypotheses, and that the first accumulator to reach threshold triggers a decision in favour of its associated hypothesis. However, the evidence is often assumed to occur as a continuous process whose origins are somewhat abstract, with no direct link to the neural signals - action potentials or 'spikes' - that must ultimately form the substrate for decision making in the brain. Here we introduce a new variant of the well-known multi-hypothesis sequential probability ratio test (MSPRT) for decision making whose evidence observations consist of the basic unit of neural signalling - the inter-spike interval (ISI) - and which is based on a new form of the likelihood function. We dub this mechanism s-MSPRT and show its precise form for a range of realistic ISI distributions with positive support. In this way we show that, at the level of spikes, the refractory period may actually facilitate shorter decision times, and that the mechanism is robust against poor choice of the hypothesized data distribution. We show that s-MSPRT performance is related to the Kullback-Leibler divergence (KLD) or information gain between ISI distributions, through which we are able to link neural signalling to psychophysical observation at the behavioural level. Thus, we find the mean information needed for a decision is constant, thereby offering an account of Hick's law (relating decision time to the number of choices). Further, the mean decision time of s-MSPRT shows a power law dependence on the KLD offering an account of Piéron's law (relating reaction time to stimulus intensity). These results show the foundations for a research programme in which spike train analysis can be made the basis for predictions about behavior in multi-alternative choice tasks.

Primary User Traffic Classification in Dynamic Spectrum Access Networks

This paper focuses on analytical studies of the primary user (PU) traffic classification problem. Observing that the gamma distribution can represent positively skewed data and exponential distribution (popular in communication networks performance analysis literature) it is considered here as the PU traffic descriptor. We investigate two PU traffic classifiers utilizing perfectly measured PU activity (busy) and inactivity (idle) periods: (i) maximum likelihood classifier (MLC) and (ii) multi-hypothesis sequential probability ratio test classifier (MSPRTC). Then, relaxing the assumption on perfect period measurement, we consider a PU traffic observation through channel sampling. For a special case of negligible probability of PU state change in between two samplings, we propose a minimum variance PU busy/idle period length estimator. Later, relaxing the assumption of the complete knowledge of the parameters of the PU period length distribution, we propose two PU traffic classification schemes: (i) estimate-then-classify (ETC), and (ii) average likelihood function (ALF) classifiers considering time domain fluctuation of the PU traffic parameters. Numerical results show that both MLC and MSPRTC are sensitive to the periods measurement errors when the distance among distribution hypotheses is small, and to the distribution parameter estimation errors when the distance among hypotheses is large. For PU traffic parameters with a partial prior knowledge of the distribution, the ETC outperforms ALF when the distance among hypotheses is small, while the opposite holds when the distance is large.

Decision-making out of neural events: from discrimination information to psychometric power laws

The strength/intensity of the stimulus in the random dot motion task (RDMT) [1] is determined by the percentage of dots in the kinematogram moving towards a saccadic target, a. Due to the uncertainty in the stimuli, neurons in sensory systems have evolved to transform environmental information, comprising evidence upon which a decision can be made (e.g. saccading to a). The neurons in the middle-temporal area (MT) appear to produce such evidence during the RDMT, given their tuning to a 'preferred' direction of visual motion. If the dots move predominantly in the preferred direction of an MT neuron, it generates inter-spike intervals (ISI) supporting a saccade to a. These ISIs seem randomly sampled from a distribution, fa, with mean, μa. Otherwise, the ISIs follow another distribution, fb, with mean, μb, where μb is larger than μa and this difference increases with stimulus strength. The accuracy vs motion-strength function of an ideal observer provided with empirical distributions like fa and fb, from a single MT neuron, approximates the subject's psychometric function (at the behavioral level) [1]. The distributions fa and fb are non-negative, positively skewed and have a mode larger than 0 (figure ​(figure1A),1A), as is typical for neural events recorded in many brain areas. Here we investigate why this is advantageous for decision-making. As theoretical decision-making units, we produced 5 new instantiations of the multi-hypothesis sequential probability ratio test (MSPRT) [2]. Each unit assumes its stream of input evidence to follow 1 of 5 probability density functions (PDF) whose compatibility with the empirical distribution of ISIs varies (figure ​(figure1A).1A). These include the Inverse Gaussian, Lognormal, Gamma, Inverse-Gamma and Exponential PDFs (the latter is the distribution of the inter-event times in the oft-used Poisson process). Under equal and appropriate conditions, we then compared their mean decision sample with that of an MSPRT instantiation that assumes Gaussian inputs proposed in [3] and discussed in general in [4], as exemplified in figure ​figure1B.1B. The mean decision sample is the mean number of observations required by a unit to identify which of N parallel information sources supports saccading to a, with a given accuracy. This decision sample is a model of the 'neural decision time'; the psychophysical reaction time also includes sensory and motor delays. The pattern of our results is explicable using a measure of the discrimination information between fa and fb, i.e. the Kullback-Leibler divergence (KLD). We found that, the mean decision sample decreases with increasing fa to fb KLD and, crucially, this follows a power law (figure ​(figure1C).1C). At the behavioral level, Pieron [5] reported the mean reaction time to the presentation of a stimulus (go/no-go decision-making) being shorter for more intense stimuli, and that a power law relates these measures. The universality of Pieron's law indicates that it can inform us of something fundamental about sensorimotor decision-making. Our results suggest that its explanation could lie in the power law relationship between the mean neural decision time and the discrimination information (KLD) among the distributions of sensory evidence. Figure 1 (A) PDFs of interest fitted to ISIs (grey bars) recorded in [1]from the MT during the RDMT. (B) mean decision sample for each MSPRT realisation (accuracy 95% over 1000 trials). (C) the values in (B) vs the fa to fb KLD. The dashed line is a fitted power ...

Sequential tests and biologically grounded multi-alternative decision making

Recently, Bogacz and Gurney [1] proposed that the cortico-basal-ganglia system implements asymptotically optimal decision making between several alternatives, based on sensory evidence, through a statistical algorithm known as the multi-hypothesis sequential probability ratio test (MSPRT). The original programme of work focused on architectural features of the system and made simplifying assumptions as to its physiology. Here, we extend that work to include more biologically-realistic properties and pathways, and explain their impact on the MSPRT performance. One assumption in [1] was that ‘noise’ on sensory signals was Gaussian distributed, which poses the problem of interpreting the signals in the negative tail of the distribution in terms of neural events. We addressed this issue in previous work [2] by deducing a new MSPRT that proposed these signals are Inverse Gaussian distributed (wholly positive and skewed). Using realistic parameters we showed this MSPRT requires about a tenth of the samples of its Gaussian predecessor to reach a decision with the same proportion of errors. A further simplification in [1] was that the processing elements were dynamics-free and the inter-element delays were zero. In reality neural membranes display non-trivial dynamics and significant inter-neuronal/synaptic processing delays are present, both having an effect on the performance. Hence we developed a system of first order differential equations with the time constants for membranes at rest available in the literature. We found that such a system had decision sample sizes roughly twice that of the non-dynamic MSPRT. However, active membranes have reduced time constants [3]. Using values one half of those at rest gave a 30% reduction in performance, and values of one fifth of those at rest, only a 4% decrease. Furthermore, following a closer examination of the anatomy, we added new pathways to give the architecture cortex→basal-ganglia→thalamus↔cortex (arrows indicating information flow), which completes the mapping of a ‘Bayesian structure’ onto it and represents the full decision/inference circuit. Closing such loop recursively feeds back the posterior inference from sampling step t as prior ‘knowledge’ for further steps t + Δt. By doing this we also constructed another recursive statistical test (also a MSPRT) that differs from the non-recursive (feedback-less) one proposed in [1]. To test the effect on performance of including these pathways, we incorporated them into a non-dynamic MSPRT with Inverse Gaussian inputs from [2], including the main delays. The recursion increased the number of samples to reach a decision by around 30% compared with its original non-recursive counterpart. This is nevertheless a small sacrifice considering that in return the system becomes more general and ultimately able to bias further decisions by adjusting its prior knowledge. Finally, in figure ​figure11 we show the evolution of several representative elements of the loop during a decision. These results suggest that the closed loop, delayed MSPRT might hold the key to explain the time course and magnitude of the firing rates observed, in macaque electrophysiological single unit recordings, during a decision task [4]. Figure 1 Time course of firing-rate-like signals on selected stages, during a single decision trial up to a threshold, using the closed loop, delayed MSPRT (1 sample every 15 ms). Compare with figure 7 in [4].

Hebbian learning in linear–nonlinear networks with tuning curves leads to near-optimal, multi-alternative decision making

Optimal performance and physically plausible mechanisms for achieving it have been completely characterized for a general class of two-alternative, free response decision making tasks, and data suggest that humans can implement the optimal procedure. The situation is more complicated when the number of alternatives is greater than two and subjects are free to respond at any time, partly due to the fact that there is no generally applicable statistical test for deciding optimally in such cases. However, here, too, analytical approximations to optimality that are physically and psychologically plausible have been analyzed. These analyses leave open questions that have begun to be addressed: (1) How are near-optimal model parameterizations learned from experience? (2) What if a continuum of decision alternatives exists? (3) How can neurons' broad tuning curves be incorporated into an optimal-performance theory? We present a possible answer to all of these questions in the form of an extremely simple, reward-modulated Hebbian learning rule by which a neural network learns to approximate the multi-hypothesis sequential probability ratio test.

Optimal decision making with biologically realistic neural signals

Action selection in animals requires rapid decision making that can discriminate the most salient requests for behavioral expression. The basal ganglia (BG) are believed to play a critical role in resolving competition between these requests [1] and, recently it has been proposed that the BG and cortex, taken together, implement a decision algorithm known as the multi-hypothesis sequential probability ratio test (MSPRT) [2]. Here, the cortex first integrates noisy ‘evidence’ indicating salience of action requests. The BG then examine this integrated evidence, and report the channel with maximal mean salience. The MSPRT is optimal in the sense that it guarantees the smallest decision time for a given error. The signals used in [2] assume cortical evidence - as neural firing rates - is supplied by Gaussian distributed signals. However, it is known that such rates are not distributed normally. To remedy this we have deduced three new forms for MSPRT that are based on more biologically realistic input signals. They include the Inverse Gaussian, Gamma and Lognormal distributions. All implementations preserve a form which could be mapped, in principle, onto the anatomy of the BG. Figure ​Figure11 shows an example of the performance of the MSPRT models across all distributions, together with the so called race model. All variants used the same input statistics (mean and variance). Counterintuitively, the relative performance of the models did not change with base level of salience or evidence. Figure 1 Performance of original Gaussian MSPRT[2], new implementations of MSPRT, and Race model. Performance is measured as decision time against number of alternatives for selection. The selection criterion was 1% error rate. The results are averaged across ... A critical parameter in the algorithm is the time step Δt used to sample the cortical input. The Gaussian, Inverse Gaussian, and Gamma cases are generated as stable Levy processes and may be scaled indefinitely with decreasing Δt. However, unlike the MSPRT with Gaussian signals, we show that decision time for the inverse Gaussian, and Gamma decreases with Δt. Figure ​Figure11 use a Δt = 1 ms for all models and clearly shows a performance advantage for the cases with more realistic distributions (the lognormal case is shown with similar scaling even though it has not an associated Levy process). Indefinitely small decision times may appear unrealistic, but we present an interpretation of cortical sampling that relates Δt to the inter-spike intervals of an ensemble of neural afferents impinging on the cortical ‘integrators’. This and, other mechanisms, provide a natural lower bound for Δt. We conclude that neurons may take advantage of the properties of natural spike trains to enhance decision making in the cortex-BG complex.

The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions

Neurophysiological studies have identified a number of brain regions critically involved in solving the problem of action selection or decision making. In the case of highly practiced tasks, these regions include cortical areas hypothesized to integrate evidence supporting alternative actions and the basal ganglia, hypothesized to act as a central switch in gating behavioral requests. However, despite our relatively detailed knowledge of basal ganglia biology and its connectivity with the cortex and numerical simulation studies demonstrating selective function, no formal theoretical framework exists that supplies an algorithmic description of these circuits. This article shows how many aspects of the anatomy and physiology of the circuit involving the cortex and basal ganglia are exactly those required to implement the computation defined by an asymptotically optimal statistical test for decision making: the multihypothesis sequential probability ratio test (MSPRT). The resulting model of basal ganglia provides a new framework for understanding the computation in the basal ganglia during decision making in highly practiced tasks. The predictions of the theory concerning the properties of particular neuronal populations are validated in existing experimental data. Further, we show that this neurobiologically grounded implementation of MSPRT outperforms other candidates for neural decision making, that it is structurally and parametrically robust, and that it can accommodate cortical mechanisms for decision making in a way that complements those in basal ganglia.

The dynamics of choice among multiple alternatives

We consider neurally based models for decision-making in the presence of noisy incoming data. The two-alternative forced-choice task has been extensively studied, and in that case it is known that mutually inhibited leaky integrators in which leakage and inhibition balance can closely approximate a drift–diffusion process that is the continuum limit of the optimal sequential probability ratio test (SPRT). Here we study the performance of neural integrators in n ⩾ 2 alternative choice tasks and relate them to a multihypothesis sequential probability ratio test (MSPRT) that is asymptotically optimal in the limit of vanishing error rates. While a simple race model can implement this ‘max-vs-next’ MSPRT, it requires an additional computational layer, while absolute threshold crossing tests do not require such a layer. Race models with absolute thresholds perform relatively poorly, but we show that a balanced leaky accumulator model with an absolute crossing criterion can approximate a ‘max-vs-ave’ test that is intermediate in performance between the absolute and max-vs-next tests. We consider free and fixed time response protocols, and show that the resulting mean reaction times under the former and decision times for fixed accuracy under the latter obey versions of Hick's law in the low error rate range, and we interpret this in terms of information gained. Specifically, we derive relationships of the forms log ( n - 1 ) , log ( n ) , or log ( n + 1 ) depending on error rates, signal-to-noise ratio, and the test itself. We focus on linearized models, but also consider nonlinear effects of neural activities (firing rates) that are bounded below and show how they modify Hick's law.