Predicting the local and global mechanical response of masonry structures or estimating their in-situ properties are critical and challenging for the design or assessment of these structures. This article presents a fast prediction/assessment model, developed using a conditional generative adversarial neural network (cGAN) to address this challenge. For the first time, the model shows to be capable of establishing a relationship between masonry microstructural features and full-field local/global mechanical response, and vice-versa, overpassing the path dependency of the non-linear mechanical problems. The strain maps and reaction forces of masonry panels are predicted at any load level from images of masonry panels, which embedded information regarding the materials properties and loading scenarios only in colours, without having any prior knowledge of the material properties and constitutive laws and without the need to access these fields in previous loading levels. Also, it is shown that the mechanical properties of masonry constituents can be predicted from full-field strain maps and loading scenarios using the developed model. This promises to become a revolutionary metamodel for the expensive finite element simulations of masonry or to support in-situ/laboratory experimental testing in the identification of masonry material properties.