Receptor activator of nuclear factor-κB ligand (RANKL) is considered the principal regulator of osteoclast differentiation. Therefore, strategies interfering with the RANKL-RANK signaling pathway may effectively inhibit osteoclast differentiation and mitigate bone resorption. Consequently, RANKL has become a promising target for new drug design strategies. Despite extensive research on specific drugs and antibodies, only a few have shown efficacy in treating osteoporosis. To address this challenge, we aimed to explore new approaches for designing drugs for osteoporosis. In this study, a 3D quantitative structure–activity relationship (QSAR) pharmacophore model was built for RANKL with reference to known inhibitor IC50 values. The optimal pharmacophore model was then employed as a 3D query to screen databases for novel lead compounds. The obtained compounds were subjected to ADMET and TOPKAT analyses to predict drug pharmacokinetics and toxicity. Molecular docking and de novo evolution approaches were applied to verify the docking binding affinities of the compounds. Five candidate compounds were subjected to further in vitro analyses to assess their anti-osteoporotic effects, among which compound 4 demonstrated significant inhibitory activity, achieving an inhibitory rate of 92.6 % on osteoclastogenesis at a concentration of 10 μM. Subsequent molecular dynamics (MD) simulations to assess the stability and behavior of compound 4 and its evolved variant, ZINC00059014397_Evo, within the RANKL binding site revealed that the variant is a potential therapeutic agent for targeting osteoclasts. This study offers valuable insights for developing next generation RANKL inhibitors for osteoporosis treatments.