This paper presents a parametric analysis examining the impact of phase change material (PCM) melting temperature, latent thermal energy storage (LTES) volume, and solar collectors’ surface area on the performance and costs of heating systems integrating a heat pump and LTES by using dynamic system simulations. The computational models, implemented in TRNSYS software, and the numerical procedure were validated by comparing the numerical results with experimental measurements. The computational models represent the entire heating systems and thus consider the mutual interaction of heat production, distribution and automatic regulation system as well as a building which represent the heat load, in transient boundary conditions. Two main system designs with different LTES roles were considered – a system in which the role of the LTES is to provide more favorable operating conditions and a system in which the LTES role is to store the heat on a sufficiently high temperature to be used directly for heating. System performance was analyzed in terms of the share of renewable energy in total delivered energy, power consumption, stored energy in LTES, seasonal performance factor, average solar collector efficiency and total annual costs. The results showed that, regarding the system in which LTES was used as a heat source for the heat pump, a better overall performance and lower costs were observed in systems with PCMs with melting temperatures in range between 19 and 29 °C, compared to systems with PCMs with melting temperatures outside this range. For system in which the heat from the LTES was used directly for heating, the highest values of system performance criteria and the lowest cost were achieved with the PCM of 54 °C melting temperature. Almost all of the considered system performance criteria and total costs were higher when the LTES volume and solar collector surface area were larger as well.