Laser-direct-drive symmetric implosions on OMEGA illuminate a target with 60 laser beams and are designed to produce spherical implosions. Each beam is smoothed using orthogonal polarizations obtained by passing the laser beams through distributed polarization rotators (DPRs). Observations of light scattered from OMEGA implosions do not show the expected symmetry and have much larger variation than standard predictions. For the first time, we have quantified the scattered-light nonuniformity from individual beams and identified the DPRs as the source of the enhanced nonuniformity. An instrument was invented that isolated and measured the variation in the intensity and polarization of the light scattered from each OMEGA beam. The asymmetric intensity and polarization measurements are explained when the on-target offsets between the two orthogonal polarizations produced by the DPRs are modeled using a 3D cross-beam energy transfer (CBET) code that tracks the polarizations of each beam. The time-integrated nonuniformity in laser absorption and scattered light due to CBET and the DPR polarization offsets during high-performance OMEGA implosions is predicted to be significant and dominated by low spherical harmonic mode numbers. The nonuniformity is predicted to be greatly reduced by replacing the DPRs with new optics that create smaller offsets.