The cubic vortical Whitham equation is a model for wave motion on a vertically sheared current of constant vorticity in a shallow inviscid fluid. It generalizes the classical Whitham equation by allowing constant vorticity and by adding a cubic nonlinear term. The inclusion of this extra nonlinear term allows the equation to admit periodic, traveling-wave solutions with larger amplitude than the Whitham equation. Increasing vorticity leads to solutions with larger amplitude as well. The stability of these solutions is examined numerically. All moderate- and large-amplitude solutions, regardless of wavelength, are found to be unstable. A formula for a stability cutoff as a function of vorticity and wavelength for small-amplitude solutions is presented. In the case with zero vorticity, small-amplitude solutions are unstable with respect to the modulational instability if kh>1.252, where k is the wavenumber and h is the mean fluid depth.