We perform three-dimensional crystal plasticity simulations of smooth and notched bar geometries made of polycrystalline hexagonal close-packed material representing a magnesium alloy. The polycrystalline microstructure is explicitly resolved to investigate the combined effect of initial texture and grain size on the rate-dependent macroscopic responses and their micromechanical underpinnings under uniaxial and multiaxial stress states. The simulations reveal that in addition to the textural effect recently investigated by Ravaji et al. (2021), grain size plays an important role in the anisotropy of macroscopic responses. For a given texture, the lateral deformation anisotropy increases with grain size refinement for all strain rates considered here. The load-deformation responses exhibit a synergistic strengthening effect in microstructures with stronger initial textures and finer grain sizes, which is enhanced with increasing notch acuity. A transition from a conventional power-law type load-deformation response to a sigmoidal load-deformation response may occur, which depends on the imposed strain rate. It is a result of the interaction between textural weakening and grain size refinement that influence extension twinning together with an equitable landscape of the different slip mechanisms. We discuss possible implications of the net material plastic anisotropy due to texture and grain size on macroscopic failure using a micromechanical basis.