With a two-dimensional structure model, the finite element method was used to simulate the chloride diffusion behavior in a heterogeneous concrete composed of two phases with distinct diffusivities, compared with its derived homogeneous medium that was given a single effective diffusion coefficient by the mean field theory. In contrast to the smooth chloride concentration profiles for the homogeneous medium, a three-dimensional concentration distribution network has been found in the hetero-structured concrete, characterized by a complicated band-like profile along the diffusion depth. Moreover, it features an undulating and narrowing width as the diffusion time and diffusion depth increase. The modeling results also manifest that the chloride diffusion in hetero-structured concretes appears to lag behind that for the homogeneous ones, showing an increasingly notable discrepancy between the two chloride concentration profiles as the diffusion proceeds. Nonetheless, such a concentration difference may be remarkably reduced when the chloride binding and time-reducing effect of diffusivity are supposed to happen within the cement paste phase of the hetero-structured concrete and its derived homogeneous medium.