Gypsum is one of the most environmentally friendly building materials and therefore its importance increases nowadays. Since the gypsum is mostly used in the form of the paste, the influence of aggregates on properties of gypsum mortars is not investigated very often. The article deals with the structure and properties of gypsum mortars with two different aggregates (silica sand and basalt). It was found, that even if the chemical composition and shape of particles of both aggregates differs significantly, the structure and properties of gypsum mortars with both types of aggregates are similar and the properties of the gypsum composite depends more on the aggregate granulometry and the quality of its surface. The aggregate with rougher surface of particles has more distinct interfacial transition zone, higher porosity and higher compressive strength.