The instability of Ekman boundary layer flow is studied inside a rotating annular cavity with radial throughflow, which is a relevant geometry of the air cooling system in turbines. The flow is computed by direct numerical simulation using a time-dependent three-dimensional Navier–Stokes solver based on a pseudo-spectral method. The fluid entering the annulus at the inner section then develops into a rotating geostrophic core flanked above and below by two nonlinear Ekman boundary layers and exits at the outer section. In this study, the rotation rate of the cavity is fixed at a given high value, corresponding to an Ekman number E=2.24×10−3. When the throughflow is weak, the motion is steady and the boundary layer flow is well described by Ekman's analytical solution. On increasing the mass flow rate, the flow becomes unsteady and perturbations appear in the form of counter-rotating pairs of vortices adjacent to upper and lower surfaces of the cavity. Multiple stable solutions, involving circular and spiral waves with different numbers of arms, are obtained at fixed mass flow rate. The wavenumber and frequency of both circular and spiral waves are determined to be characteristic of the type II viscous Ekman layer instability.