Efforts to couple mantle flow models with rheological theories of mineral deformation typically ignore the effect of texture development on flow evolution. The fact that there are only three easy slip systems for dislocation glide in olivine crystals leads to strong mechanical interactions between the grains as the deformation proceeds, and subsequent development of large viscoplastic anisotropy in polycrystals exhibiting pronounced Lattice Preferred Orientations. Using full‐field simulations for creep in dry polycrystalline olivine at high temperature and low pressure, it is shown that very large stress and strain rate intragranular heterogeneities can build up with deformation, which increase dramatically with the strength of the hard slip system (included for the purpose of enabling general deformations). Compared with earlier nonlinear extensions of the Self‐Consistent mean‐field theory to simulate polycrystal deformation, the “Second‐Order” method is the only one capable of accurately describing the effect of intraphase stress heterogeneities on the macroscopic flow stress, as well as on the local stress‐ and strain rate fluctuations in the material. In particular, this approach correctly predicts that olivine polycrystals can deform with only four independent slip systems. The resistance of the fourth system (or accommodation mechanism), which is likely provided by dislocation climb or grain boundary processes as has been observed experimentally, may essentially determine the flow stress of olivine polycrystals. We further show that the “tangent” model, which had been used extensively in prior geophysical studies of the mantle, departs significantly from the full‐field reference solutions.