Natural minerals and earths with coloring properties have been widely used as artistic pigments since prehistoric times. Despite being extensively studied, the complex chemistry of earth pigments is still unsatisfactory described with respect to their mineralogical and structural variability and origin. In this study, a large group of earth pigments from various geographical locations was investigated using easily accessible spectroscopic techniques and multivariate analysis with the aim to identify distinctive mineralogical and chemical characteristics of natural pigment sources. Portable X-ray fluorescence (p-XRF), Fourier transform infrared spectroscopy (FTIR) and fiberoptic Raman spectroscopy were used for the elemental, molecular and structural characterization of the investigated pigments. Diagnostic spectral features and chemical patterns (fingerprints) were identified and discussed with respect to their geological sources. Due to the occurrence of similar accompanying minerals, it was observed that the differentiation of red and yellow ochers is more challenging compared to green, brown and black pigments. However, for some of the investigated pigments, the presence of certain accessory minerals and/or of certain chemical patterns can have diagnostic value. Principal component analysis (PCA) of the FTIR and XRF data matrices showed promising results in terms of geological attribution, highlighting a promising tool for provenance research. The results of the study demonstrate the potential benefits of this rapid and nondestructive approach for the characterization and differentiation of earth pigments with similar hues coming from different geological sources.