With the industry development, materials that combine good mechanical properties with new functional applications are in high demand. Here, a solution was provided to overcome low mechanical properties of soft magnetic materials through heterostructure design in CoFeNi high-entropy alloys (HEA). A CoFeNi-based HEA with a hierarchical structure with heterogeneous microstructure, consisting of body-centered cubic (BCC) bands, ultra-fine grained dual-phase and non-recrystallized lamellae (NRL) zones containing a high-density of nanoprecipitates, was fabricated and finely tuned by cold rolling (CR) and annealing. The HEA annealed at 850 °C (HEA-850) exhibits a noteworthy strength-ductility synergy than the counterparts with homogeneous dual-phase microstructures. The yield strength and elongation of the HEA-850 sample are nearly up to 1 GPa and 24%, respectively. Furthermore, because of the elimination of high-density dislocations, this alloy possesses good soft magnetic characteristics, and the saturation magnetization (Ms) and coercivity are 120 emu/g and 455 A/m, respectively. Detailed microstructure observations reveal that the high yield strength is mainly attributed to the interphase boundary strengthening from the dual-phase ultra-fine regions and nanoprecipitate strengthening as well as dislocation strengthening from NRL regions. Due to the mechanical and microstructural heterogeneity, multiple hetero-deformation induced hardening sustains persistent work hardening during tensile deformation, resulting in the reasonable ductility in the HEA-850 sample. The novel provides an important insight into developing high-performance materials with a superior combination of mechanical and functional properties using heterogeneous microstructure.