Integrated reference electrodes allow to deconvolute voltage contributions of anode and cathode and contribute to a better understanding of CO2 electrolyzers. However, in zero-gap cell configurations, this integration can be challenging and obtaining error-free data with such a setup is a non-trivial task. This study compares five different methods to integrate a reference electrode into an alkaline zero-gap CO2 electrolysis cell. Sources of error and measures to circumvent them are investigated and finite-element simulation is used to gain a better understanding of observed effects. Placing a reference electrode into the inactive area of the cell is found to be a reliable method, as long as the placement of electrodes is sufficiently controlled. Sandwiching a wire quasi-reference electrode between two membranes is especially useful for electrochemical impedance spectroscopy; however, it can affect the overall cell performance. Contacting the catalyst layer from the backside with a salt-bridge is promising for localized measurements if sufficient reproducibility can be ensured.