Consumer perception and market demand have driven the replacement of synthetic colorants with naturally derived alternatives in the cosmetic industry. This study describes a facile way to prepare durable inorganic-organic hybrid pigment with advanced biocompatibility, antibacterial and hydrophobic properties tailored for color cosmetics by initial modification of palygorskite with chitosan to anchor sappanwood dye extract and subsequently coating with amino-modified silicone oil (ASO). The hybrid pigments were characterized by transmittance electron microscopy, X-ray diffraction, Fourier transform infrared spectroscopy, X-ray photoelectron spectroscopy and the Brunauer–Emmett–Teller method. The results indicated that the sappanwood dye was loaded on chitosan-modified palygorskite via hydrogen bonding and electrostatic interaction. Furthermore, the chitosan-palygorskite/sappanwood hybrid pigment exhibited enhanced biocompatibility and color stability on exposure to different heating temperatures and UVA radiation after subsequent hydrophobic modification with amino-modified silicone oil. Moreover, facial foundation cosmetics based on the chitosan-palygorskite/sappanwood@ASO composites exhibited excellent brightening and skin color corrective effect on human volunteers without any adverse response. And no significant difference was observed in 12 out of 14 sensory evaluation indexes in the comparison of this hybrid pigment-based makeup with two commercially available products. This study provides a new route to stabilize natural botanical colorant for cosmetic use by chitosan-modified clay minerals.