To obtain spatial information about an arbitrary atomic distribution in x-ray structure analysis, e.g. in molecules or proteins, the standard method is to measure the intensity in the far field, i.e. the first-order photon correlation function of the coherently scattered x-ray photons (coherent diffractive imaging). Recently, it was suggested to record alternatively the incoherently scattered photons and measure the second-order photon correlation function to reconstruct the geometry of the unknown atomic distribution (incoherent diffractive imaging). Yet, besides various advantages of the latter method, both techniques suffer from the so-called phase retrieval problem. Lately, an ab-initio phase retrieval algorithm to reconstruct the phase of the so-called structure factor of the scattering objects based on the third-order photon correlation function was reported. The algorithm makes use of the closure phase, which contains important, yet incomplete phase information, well-known from triple correlations and their bispectrum in speckle masking and astronomy applications. Here, we provide a detailed analysis of the underlying scheme and quantities in the context of x-ray structure analysis. In particular, we explicitly calculate for the first time the third-order photon correlation function for single photon emitters in a full quantum mechanical treatment and discuss the uniqueness of the closure phase equations constructed from. In this context, we recapitulate the sign problem of the closure phase and how it can be lifted using redundant information. We further show how the algorithm can be improved using even higher-order photon correlation functions produced by single photon emitters, e.g. the fourth-order correlation function, delivering new phase relations appearing in the four-point correlations.