Large Sample Size Limit Research Articles

Approximate filtering of conditional intensity process for Poisson count data: Application to urban crime

The primary focus is a sequential data assimilation method for count data modelled by an inhomogeneous Poisson process. In particular, a quadratic approximation technique similar to the extended Kalman filter is applied to develop a sub-optimal, discrete-time, filtering algorithm, called the extended Poisson–Kalman filter (ExPKF), where only the mean and covariance are sequentially updated using count data via the Poisson likelihood function. The performance of ExPKF is investigated in several synthetic experiments where the true solution is known. In numerical examples, ExPKF provides a good estimate of the “true” posterior mean, which can be well-approximated by the particle filter (PF) algorithm in the very large sample size limit. In addition, the experiments demonstrate that the ExPKF algorithm can be conveniently used to track parameter changes; on the other hand, a non-filtering framework such as a maximum likelihood estimation (MLE) would require a statistical test for change points or implement time-varying parameters. Finally, to demonstrate the model on real-world data, the ExPKF is used to approximate the uncertainty of urban crime intensity and parameters for self-exciting crime models. The Chicago Police Department’s CLEAR (Citizen Law Enforcement Analysis and Reporting) system data is used as a case study for both univariate and multivariate Hawkes models. An improved goodness of fit measured by the Kolmogorov–Smirnov (KS) statistics is achieved by the filtered intensity. The potential of using filtered intensity to improve police patrolling prioritisation is also tested. By comparing with the prioritisation based on MLE-derived intensity and historical frequency, the result suggests an insignificant difference between them. While the filter is developed and tested in the context of urban crime, it has the potential to make a contribution to data assimilation in other application areas.

Single-hole wave function in two dimensions: A case study of the doped Mott insulator

We study a ground-state ansatz for the single-hole doped $t$-$J$ model in two dimensions via a variational Monte Carlo (VMC) method. Such a single-hole wave function possesses finite angular momenta generated by hidden spin currents, which give rise to a novel ground state degeneracy in agreement with recent exact diagonalization (ED) and density matrix renormalization group (DMGR) results. We further show that the wave function can be decomposed into a quasiparticle component and an incoherent momentum distribution in excellent agreement with the DMRG results up to an $8\times 8 $ lattice. Such a two-component structure indicates the breakdown of Landau's one-to-one correspondence principle, and in particular, the quasiparticle spectral weight vanishes by a power law in the large sample-size limit. By contrast, turning off the phase string induced by the hole hopping in the so-called $\sigma\cdot t\text{-}J$ model, a conventional Bloch-wave wave function with a finite quasiparticle spectral weight can be recovered, also in agreement with the ED and DMRG results. The present study shows that a singular effect already takes place in the single-hole-doped Mott insulator, by which the bare hole is turned into a non-Landau quasiparticle with translational symmetry breaking. Generalizations to pairing and finite doping are briefly discussed.

Convergence of posteriors for structurally non-identified problems using results from the theory of inverse problems

Abstract. We consider convergence of the posterior distribution in a Bayesian parameter estimation framework in the large sample size limit for structurally non-identified problems. These belong to the class of ill-posed problems, and the large sample theory is not applicable here. In particular, the influence of the prior distribution does not vanish in the large sample size limit. We review recent results in this area and present ideas inspired from the theory of ill-posed inverse problems that can be used towards a more general concept of posterior convergence for non-identified problems.

Opportunistic Detection Under a Fixed-Sample-Size Setting

With a finite number of samples drawn from one of two possible distributions sequentially revealed, an opportunistic detection rule is proposed, which possibly makes an early decision in favor of the alternative hypothesis, while always deferring the decision of the null hypothesis until collecting all the samples. Properties of this opportunistic detection rule are discussed and its key asymptotic behavior in the large sample size limit is established. Specifically, a Chernoff-Stein lemma type of characterization of the exponential decay rate of the miss probability under the Neyman-Pearson criterion is established, and consequently, a performance metric of asymptotic exponential efficiency loss is proposed and discussed, which is exactly the ratio between the Kullback-Leibler distance and the Chernoff information of the two hypotheses. Analytical results are corroborated by numerical experiments.

Large System Spectral Analysis of Covariance Matrix Estimation

Eigendecomposition of estimated covariance matrices is a basic signal processing technique arising in a number of applications, including direction-of-arrival estimation, power allocation in multiple-input/multiple-output (MIMO) transmission systems, and adaptive multiuser detection. This paper uses the theory of non-crossing partitions to develop explicit asymptotic expressions for the moments of the eigenvalues of estimated covariance matrices, in the large system asymptote as the vector dimension and the dimension of signal space both increase without bound, while their ratio remains finite and nonzero. The asymptotic eigenvalue distribution is also obtained from these eigenvalue moments and the Stieltjes transform, and is extended to first-order approximation in the large sample-size limit. Numerical simulations are used to demonstrate that these asymptotic results provide good approximations for finite systems of moderate size.

Sample Eigenvalue Based Detection of High-Dimensional Signals in White Noise Using Relatively Few Samples

The detection and estimation of signals in noisy, limited data is a problem of interest to many scientific and engineering communities. We present a mathematically justifiable, computationally simple, sample-eigenvalue-based procedure for estimating the number of high-dimensional signals in white noise using relatively few samples. The main motivation for considering a sample-eigenvalue-based scheme is the computational simplicity and the robustness to eigenvector modelling errors which can adversely impact the performance of estimators that exploit information in the sample eigenvectors. There is, however, a price we pay by discarding the information in the sample eigenvectors; we highlight a fundamental asymptotic limit of sample-eigenvalue-based detection of weak or closely spaced high-dimensional signals from a limited sample size. This motivates our heuristic definition of the effective number of identifiable signals which is equal to the number of ldquosignalrdquo eigenvalues of the population covariance matrix which exceed the noise variance by a factor strictly greater than . The fundamental asymptotic limit brings into sharp focus why, when there are too few samples available so that the effective number of signals is less than the actual number of signals, underestimation of the model order is unavoidable (in an asymptotic sense) when using any sample-eigenvalue-based detection scheme, including the one proposed herein. The analysis reveals why adding more sensors can only exacerbate the situation. Numerical simulations are used to demonstrate that the proposed estimator, like Wax and Kailath's MDL-based estimator, consistently estimates the true number of signals in the dimension fixed, large sample size limit and the effective number of identifiable signals, unlike Wax and Kailath's MDL-based estimator, in the large dimension, (relatively) large sample size limit.

Analytical prediction of sample eigenvector quality deterioration in large arrays due to SNR or sample size constraints

It is well-known that subspace-based estimation methods in adaptive array processing suffer a rapid degradation in performance as either the signal-to-noise ratio (SNR) or the number of available snapshots drops below a certain threshold value. In the large system, relative large sample size limit, one can use random matrix theory to analytically predict this threshold and the degradation in the ''quality'' of the corresponding subspace estimates. In certain settings, one observes a "phase transition" phenonemon so that if the signals are too weak or there are insufficient number of snapshots or both, the subspace estimates are, statistically speaking, noise-like. We discuss the implication of these results for the subspace based detection of signals in white and colored noise using large arrays and illustrate the accuracy of the predictions with numerical simulations.