Inhibition of SIRT3 exhibited potency in triggering leukemic cell differentiation. In discovery of potent SIRT3 inhibitors for cancer differentiation therapy, structural modification was performed on the previously developed lead compound P6. A total of 33 compounds were designed and synthesized. In the enzyme inhibitory assay, several molecules S18, S26, S27 and T5 showed potent SIRT3 inhibitory activity with IC50 value of 0.53, 1.86, 5.06, and 2.88 μM, respectively. Moreover, the tested compounds exhibited SIRT3 inhibitory selectivity over SIRT1 and SIRT2. Compounds S27 and T5 were potent in inhibition the growth of MM1.S and RPMI-8226 cells in the invitro antiproliferative test. Significantly, representative compounds, especially S27 and T5, promoted differentiation of tested MM cells in the cellular morphological evaluation, accompanied by increasing the expression of differentiation antigen CD49e and human immunoglobulin light chain lambda and kappa. Additionally, molecule S18 without antiproliferative potency itself, showed significant inhibitory activity against growth factor IL-6 induced RPMI-8226 cell proliferation. Collectively, potent SIRT3 selective inhibitors with MM cell differentiation potency were developed for further discovery of anticancer drugs.