The interaction between drilling fluids and oil and gas reservoir formations can result in the reduction or blockage of the micropore structural network, leading to a consequent decrease in permeability. Due to the negative economic impact of formation damage on hydrocarbon production, formulating drilling fluids with characteristics that minimise formation damage has been a major focus of the petroleum industry and researchers since the 1970s–1980s. Formation damage can significantly affect the ultimate productivity in hydrocarbon-bearing formations, especially in sandstone reservoirs with varying clay mineralogy such as kaolinite, smectite, illite, and mixed layer (illite + smectite) clay mineralogy in the micropore structural networks. These clay minerals are highly unstable/reactive and often appear as pore lining, pore bridging, and pore-filling along the walls of the micropore structural, inevitably coming into contact with drilling fluids (both solids and filtrates) that have invaded the micropore structure. Existing studies on the understanding of the relationship between drilling fluid formulation and formation damage mechanisms have not adequately addressed the role of individual species of clay minerals and their varying physicochemical properties during the formulation of drilling fluids for sandstone reservoirs with varying clay minerals. Instead, clay minerals are often treated as a whole or a single group. The work presented in this paper aims to examine and quantifies the impact of three drilling fluids (PAC-UL water-based, potassium formate, and halide drilling fluids) on productivity impairment in kaolinitic sandstone reservoirs and in kaolinitic and mixed-layer (illitic + smectitic) sandstone reservoirs. To achieve this, a high-temperature, high-pressure (HTHP) formation damage testing facility was designed and built to determine the initial and return permeability of selected hydrocarbon reservoirs using core samples. The formation damage mechanisms resulting from the interaction of the reservoir rocks with different formulations of drilling fluids are fully characterized using a combination of scanning electron microscopy, energy dispersive X-ray and micro-computed tomography. For all three drilling fluid formulations, the results showed an enhanced return permeability of the kaolinitic sandstone after interaction with the specially formulated halide drilling fluid.