Non-Newtonian fluids are of interest in industrial sectors, biological problems and other natural phenomena. This work proposes rheologically-dependent, spatially and temporally high-order non-residual stabilized finite element formulations. The accuracy of the methods is assessed by tackling highly-convective time-dependent power-law flows. The spatial approximation uses Lagrangian finite elements up to fourth order. The temporal integration is done via backward differentiation formulas of order one, two and three. A key aspect of our work is using a non-residual orthogonal variational multiscale (VMS) formulation to stabilize dominant convection and to allow equal-order interpolation of velocity and pressure. Our VMS method uses dynamic nonlinear subscales, which have not been tested so far for generalized Newtonian fluids. In this work, the use of high-order temporal discretizations for the subscale components is systematically evaluated. Numerical experiments consider the flow over a confined cylinder for Reynolds numbers between 40 and 400 and power-law indices between 0.4 and 1.8. Numerical testing demonstrates the method to be stable in all combinations of spatial and temporal orders. Our results show that using high-order spatial discretizations more accurately approximates boundary layers and viscosity fields. Moreover, higher temporal orders allow using larger time steps while still capturing highly dynamic behaviors with better resolution in frequency spectra.