Kerr-effect-induced changes of the polarization state of light are well known in pulsed laser systems. An example is nonlinear polarization rotation, which is critical to the operation of many types of mode-locked lasers. Here, we demonstrate that the Kerr effect in a high-finesse Fabry-Pérot resonator can be utilized to control the polarization of a continuous wave laser. It is shown that a linearly-polarized input field is converted into a left- or right-circularly-polarized field, controlled via the optical power. The observations are explained by Kerr-nonlinearity induced symmetry breaking, which splits the resonance frequencies of degenerate modes with opposite polarization handedness in an otherwise symmetric resonator. The all-optical polarization control is demonstrated at threshold powers down to 7 mW. The physical principle of such Kerr effect-based polarization controllers is generic to high-Q Kerr-nonlinear resonators and could also be implemented in photonic integrated circuits. Beyond polarization control, the spontaneous symmetry breaking of polarization states could be used for polarization filters or highly sensitive polarization sensors when operating close to the symmetry-breaking point.