Recovery and separation of rare earth elements was highly crucial but difficult. In this study, a well-designed new ionic liquid named as benzyltributylammonium decanedioate ([N444Bn]2[SA]) was developed to extract rare earths and separate Sm(III) from the spent SmCo magnets in acetate medium for the first time. To exhibit the superiority of acetate medium, the extraction of Nd(III) and La(III) in three different media were comparatively investigated under different conditions, including equilibrium time, initial pH of aqueous solution, rare earth concentration, extractant concentration, salting-out agent concentration and extraction temperature. The results revealed that the better extractabilities of La(III) were observed in acetate medium rather than those in chloride medium (i.e., 97.6% versus 78.6%), which may be attributed to the low-hydrated nature of acetate ions. Importantly, the back-extraction experiments indicated that 91.4% Nd(III) can be stripped from [N444Bn]2[SA] by using 1.5 mol/L HOAc, revealing the outstanding reusability of ionic liquid in the industrial separation-stripping cycles. Furthermore, the ion association mechanism was proposed to describe the extraction behavior and confirmed by the slope analysis and FT-IR characterizations in three media. In terms of the SmCoCu mixture separation, the maximal separation factor of Sm(III)/Co(II) was calculated as 3078 in simulated solution. Impressively, the recovery of Sm(III) from the actual SmCo magnets by our dicarboxylate-type ionic liquid was also successfully achieved with a recycling rate of 96.8% and a Sm(III)/Co(II) selectivity of 148, which exhibited the potential application in the separation of Sm(III) from the spent SmCo magnets.