To address the dilemma of the stiffness and toughness properties of high-density polyethylene (HDPE) composites, titanate coupling agent-treated CaCO3 nanoparticles (nano-CaCO3) and ethylene-octene copolymer (POE) were utilized to blend with HDPE to prepare ternary nanocomposites via a two-sequence-step process. Meanwhile, a one-step process was also studied as a control. The obtained ternary nanocomposites were characterized by scanning electron microscopy (SEM), Advanced Rheometrics Expansion System (ARES), Dynamic Mechanical Analysis (DMA), wide-angle X-ray diffraction analysis (WXRD), and mechanical test. The SEM results showed one or two CaCO3 nanoparticles were well-encapsulated by POE and were uniformly dispersed into the HDPE matrix to form a core-shell structure of 100-200 nm in size by the two-step process, while CaCO3 nanoparticles were aggregated in the HDPE matrix by the one-step method. The result of the XRD showed that the nano-CaCO3 particle played a role in promoting crystallization in HDPE nanocomposites. Mechanical tests showed that the synergistic effect of both the POE elastomer and CaCO3 nanoparticles should account for the balanced performance of the ternary composites. In comparison with neat HDPE, the notched impact toughness of the ternary nanocomposites of HDPE/POE/nano-CaCO3 was significantly increased. In addition, the core-shell structure absorbed the fracture impact energy and prevent further propagation of micro-cracks, thus obtaining a higher notched Izod impact strength.