We study the linear stability of weakly magnetized differentially rotating plasmas in both collisionless kinetic theory and Braginskii's theory of collisional, magnetized plasmas. We focus on the very weakly magnetized limit that is important for understanding how astrophysical magnetic fields originate and are amplified at high redshift. We show that the single instability of fluid theory - the magnetorotational instability mediated by magnetic tension - is replaced by two distinct instabilities, one associated with ions and one with electrons. Each of these has a different way of tapping into the free energy of differential rotation. The ion instability is driven by viscous transport of momentum across magnetic field lines due to a finite ion cyclotron frequency (gyroviscosity); the fastest growing modes have wavelengths significantly longer than MHD and Hall MHD predictions. The electron instability is a whistler mode driven unstable by the temperature anisotropy generated by differential rotation; the growth time can be orders of magnitude shorter than the rotation period. The electron instability is an example of a broader class of instabilities that tap into the free energy of differential rotation or shear via the temperature anisotropy they generate. We briefly discuss the application of our results to the stability of planar shear flows and show that such flows are linearly overstable in the presence of fluid gyroviscosity. We also briefly describe the implications of our results for magnetic field amplification in the virialized halos of high redshift galaxies.