Abstract. The objective of the Earth Cloud, Aerosol, and Radiation Explorer (EarthCARE) mission is to infer attributes of cloud, aerosol, precipitation, and radiation from observations made by four complementary instruments. This requires the development of single-instrument and multiple-instrument (i.e. synergistic) retrieval algorithms that employ measurements made by one, or more, of EarthCARE's cloud-profiling radar (CPR), atmospheric lidar (ATLID), and multi-spectral imager (MSI); its broadband radiometer (BBR) places the retrieved quantities in the context of the surface–atmosphere radiation budget. To facilitate the development and evaluation of ESA's EarthCARE production model prior to launch, sophisticated instrument simulators were developed to produce realistic synthetic EarthCARE measurements for simulated conditions provided by cloud-resolving models. While acknowledging that the physical and radiative representations of cloud, aerosol, and precipitation in the test scenes are based on numerical models, the opportunity to perform detailed evaluations wherein the “truth” is known provides insights into the performance of EarthCARE's instruments and retrieval algorithms. This level of omniscience will not be available for the evaluation of in-flight EarthCARE retrieval products, even during validation activities coordinated with ground-based and airborne measurements. In this study, we compare EarthCARE retrieval products both statistically across all simulated scenes and from a specific time series from a single scene. For ice clouds, it is shown that retrieved profiles of ice water content and effective particle size made by the ATLID-CPR-MSI cloud, aerosols, and precipitation (ACM-CAP) synergistic algorithm are consistently more accurate than those from its single-instrument counterparts. While liquid clouds are often difficult to detect from satellite-borne sensors, especially for multi-layered clouds, ACM-CAP benefits from combined constraints from lidar backscatter, solar radiances, and radar-path-integrated attenuation but still exhibits non-trivial random error. For precipitation retrievals, the CPR cloud and precipitation product (C-CLD) and ACM-CAP have a similar performance when well-constrained by CPR measurements. The greatest differences are in coverage, with ACM-CAP reporting retrievals in the melting layer, and in heavy precipitation, where CPR signals are dominated by multiple scattering and attenuation. Aerosol retrievals from ATLID compensate for a high degree of measurement noise in a number of ways, with the ATLID extinction, backscatter, and depolarisation (A-EBD) product and ACM-CAP demonstrating similar performance. The multi-spectral imager (MSI) cloud optical properties (M-COP) product performs very well for unambiguous cloud layers. Similarly, the MSI aerosol optical thickness (M-AOT) product performs well when radiances are unaffected by cloud, but both products provide little information about vertical profiles of properties. Finally, a summary of the performance of all retrieval products and their random errors is provided.