On the approximation of the integer least-sqaures success rate: which lower or upper bound to use?

Abstract

The probability of correct integer estimation, the success rate, is an important measure when the goal is fast and high precision positioning with a Global Navigation Satellite System. Integer ambiguity estimation is the process of mapping the least-squares ambiguity estimates, referred to as the float ambiguities, to an integer value. It is namely known that the carrier phase ambiguities are integer-valued, and it is only after resolution of these parameters that the carrier phase observations start to behave as very precise pseudorange measurements. The success rate equals the integral of the probability density function of the float ambiguities over the pull-in region centered at the true integer, which is the region in which all real values are mapped to this integer. The success rate can thus be computed without actual data and is very valuable as an a priori decision parameter whether successful ambiguity resolution is feasible or not. The pull-in region is determined by the integer estimator that is used and therefore the success rate also depends on the choice of the integer estimator. It is known that the integer least-squares estimator results in the maximum success rate. Unfortunately, it is very complex to evaluate the integral when integer least-squares is applied. Therefore, approximations have to be used. In practice, for example, the success rate of integer bootstrapping is often used as a lower bound. But more approximations have been proposed which are known to be either a lower or upper bound of the actual integer least-squares success rate. In this contribution an overview of the most important lower and upper bounds will be given. These bounds are compared theoretically as well as based on their performance. The performance is evaluated using simulations, since it is then possible to compute the ’actual’ success rate. Simulations are carried out for the two-dimensional case, since its simplicity makes evaluation easy, but also for the higher-dimensional geometry-based case, since this gives an insight to the performance that can be expected in practice.