Stone-building materials, despite their natural origin, must be tested for the concentration of trace elements necessary to assess their impact on the environment and humans. In addition to basic research determining their mineral composition and structural and textural features, it is important to analyze the geochemical interactions between the material matrix and the concentration of elements that have a negative impact on the surrounding natural environment and our health. In the presented study, mineralogical and geochemical studies were carried out on the Carpathian sandstones. It was shown that the studied sandstones are represented by lithic wackes and sublithic arenites. Rocks subject to the secondary process of sulfide mineralization were observed among sublithic arenites. Pyrite in the studied geomaterials took various forms. A detailed geochemical analysis was carried out in the material in which iron sulfides acted as a binder. The research was aimed at identifying possible variations in the concentration of elements, with a particular emphasis on the contact between the silica and mineralized phases. The assessment of the geochemical interaction of iron sulfides with silica at a successively enlarged measurement was carried out using the Mamdani–Assilian fuzzy inference model.