A phase field model was developed to study the relationships between particulate microstructure and the magnetoelectric properties of magnetostrictive–piezoelectric composites. The model explicitly treats grain structures and domain processes in two-phase composites and automatically takes into account boundary conditions at inter-phase interfaces and grain boundaries. The model was used to investigate certain fundamental issues important for both rational design and practical application of magnetoelectric composites, such as connectivity of constituent phases, electrical conductivity of the magnetostrictive phase, in situ electric poling of ferroelectric phase within composite and the bias magnetic field during sensing. The evolution of magnetic and electric domains and their associated magnetostriction and electrostriction were simulated to study the strain-mediated domain-level coupling mechanisms that exist between magnetization and polarization. The simulations show that in situ poling of the ferroelectric phase within the composite microstructure is essential for piezoelectricity and thus the magnetoelectric effect, which requires 0–3 connectivity of the magnetostrictive–ferroelectric microstructure. Contrary to conventional wisdom that regards the conductivity of the magnetostrictive phase as detrimental to the magnetoelectric effect, it was found that a finite electrical conductivity of an isolated magnetostrictive phase can significantly enhance the magnetoelectric coefficient by both assisting effective poling and helping to stabilize the poled ferroelectric state. The simulations also show that the polarization response markedly lags behind the magnetization response, and a bias magnetic field effectively enhances the magnetoelectric effect.