Light produced by most natural and artificial sources is unpolarized or partially polarized. At any instant of time, however, such random light can be regarded as fully polarized, but the polarization state may vary drastically within short time intervals. This rate of change is another attribute that separates one unpolarized beam from another. Here, we study such polarization dynamics and, for the first time to our knowledge, measure the characteristic time, called the polarization time, in which the instantaneous polarization state stays essentially unaltered. The technique employs a polarization-sensitive Michelson interferometer and two-photon absorption detection valid for electromagnetic light, yielding superior femtosecond time resolution. We analyze two unpolarized light sources: amplified spontaneous emission from a fiber amplifier and a dual-wavelength laser source. The characterization of polarization dynamics can have significant applications in optical sensing, polarimetry, telecommunication, and astronomy, as well as in quantum and atom optics.