On the suitability of using the diagonal Gaussian approximation in the calculation of the magnitude-based likelihood function.

Abstract

Statistical likelihood maximization is currently one of the main tools in computational procedures in biological crystallography. In these procedures, the likelihood function is calculated, as a rule, within the framework of a diagonal Gaussian approximation (DGA) of the joint probability distribution of the real and imaginary parts of a set of structure factors. This approximation assumes pairwise uncorrelated values of various structure-factor components. In this paper, exact formulas are derived for pairwise correlations of structure factors, and conditions under which these correlations can be considered to be negligible are discussed. It is shown that in the case where the probability distribution of the atomic coordinates is related to the region of the molecule or its domains, the correlation of the structure factors of reflections s and w is determined mostly by the magnitudes of the Fourier transform of the probability distribution calculated at the points 2s, 2w, s - w and s + w. However, in the case where the probability distribution describes small corrections to the coordinates of the existing preliminary atomic model, the correlation is determined by the values of the structure factors of the preliminary model that correspond to the 2s, 2w, s - w and s + w reflections rather than by the Fourier transform of the probability distribution. Test cases demonstrate that the practice of using the DGA for calculation of the likelihood when based on sets containing neighbouring reflections may be unjustified in some crystallographic applications, especially in single-particle studies.