A novel rosin-based ester tertiary amine (RETA) with three hydrophilic groups and a rigid hydrophobic group was synthesized from rosin by Diels-Alder addition, acylation and esterification reactions. RETA was characterized by infrared spectroscopy (FT-IR) and proton nuclear magnetic resonance spectroscopy (13C NMR). Results from testing surface tension, zeta potential, and transmission electron spectroscopy showed that RETA had unique pH responsiveness. RETA self-assembled into worm-like micelles, spherical micelles 130 nm in diameter and big spherical worm-like aggregates with diameter of 2 μm at pH = 5.76, 8.04 and 9.38, respectively. The critical micelle concentration (CMC) of RETA was 0.42 mmol/L, and the surface tension at CMC (γcmc) was 38.73 mN/m when pH was 8.04. The RETA had a potential application in delivering doxorubicin hydrochloride (DOX) due to the pH responsiveness. Self-assembly mixed systems of RETA and rosin-based phosphoric acid (DDPD) were designed to improve emulsification. The mixed systems had obvious synergistic effects and unexpected emulsification. The γcmc and CMC of mixtures were 41.74 mN/m and 0.20 mmol/L, the size of mixture micelles increased up to 300 nm in the optimum molar ratio of RETA/DDPD (7:3) by TEM and cryo-TEM. It was worth noting that the mixture system formed vesicles in the RETA/DDPD molar ratio of 5:5. The stability time of emulsion with RETA and DDPD as emulsifier were only 63 s and 52 s respectively, but the stability time increased to 234 s in the optimum molar ratio. In addition, the formation mechanisms of micelles at different pH and in various mixtures were discussed in detail. What's more, cytotoxicity results showed that the toxicity of RETA was lower significantly than that of lecithin, a food ingredient in egg yolk and soybean. The cell viability was more than 83% in the high concentration of RETA (4000 μg/ml).