Both cyclopropyl amide and piperazine sulfonamide functional groups are known for their various biological properties used for drug development. Herein, we synthesized nine new derivatives with different substituent groups incorporating these moieties and screened them for their anti-osteoclast differentiation activity. After analyzing the structure–activity relationship (SAR), the inhibitory effect against osteoclastogenesis was determined to be dependent on the lipophilicity of the compound. Derivative 5b emerged as the most effective dose-dependent inhibitor after TRAP staining with an IC50 of 0.64 µM against RANKL-induced osteoclast cells. 5b was also able to suppress F-acting ring formation and bone resorption activity of osteoclasts in vitro. Finally, well-acknowledged gene and protein osteoclast-specific marker expression levels were decreased after 5b administration on primary murine osteoclast cells.