By analyzing the decaying intensity, leaking out a high-finesse cavity previously ``filled'' by a cw laser source (using the cavity ring-down spectroscopy technique), we observed frequency beating between what we think are two orthogonal eigenpolarization states of the intracavity electromagnetic field. The time decay (ring down) is analyzed by varying the angle of the polarization analyzer located in front of the detector. A full modeling of the observed signal is proposed. It is based on the Jones matrix formalism required for modeling the cavity behavior following a rotated phase shifter. The full transfer function is first established in the frequency domain, and then Fourier transformed to recover the temporal response. The same optical cavity, i.e., constituted of the same set of mirrors, is used at two different wavelengths ($\ensuremath{\sim}800$ and $\ensuremath{\sim}880$ nm). It demonstrates the differences in behavior between a high-finesse cavity $(\ensuremath{\sim}400\phantom{\rule{0.16em}{0ex}}000)$ and a lower finesse cavity $(\ensuremath{\sim}50\phantom{\rule{0.16em}{0ex}}000)$. Beating frequency, characteristics time, and beat amplitude are mainly discussed versus the analyzer angle. A cavity birefringence of $\ensuremath{\sim}1.6\ifmmode\times\else\texttimes\fi{}{10}^{\ensuremath{-}5}$ rad, resulting from the mirror birefringence is suggested. If the current analysis is in agreement with pulsed CRDS experiments (polarimetry) obtained in an isotropic moderate-finesse cavity, it differs from a recent work report on a high-finesse cavity associated with a source mode locking [Phys. Rev. A 85, 013837 (2012)].