A class of ZN≥3-symmetric WIMP dark matter models that are characterized by the semi-annihilation into the companion of dark matter has been proposed in ref. [1], providing a mechanism to evade the stringent direct detection constraint. In this work, we point out that such models naturally provide the three Sakharov elements necessary for dark matter asymmetry, and moreover this asymmetry can be transferred to the visible sector with a proper link to the leptonic or quark sector. In our minimal Z3 example, the migration to the leptonic sector is via the asymmetric companion decay into neutrinos, and the lepton asymmetry can be further transferred to the quark sector. The CP violation parameter is restrained in this model. Thus, we explore the thermal motion effect of dark matter and find that it gives an enhancement to the CP violation parameter, which is studied for the first time. A preliminary numerical analysis based on the Boltzmann equations shows that both correct relic density of dark matter and baryon asymmetry can be accommodated.