Correctly estimating the wind stress at the sea surface is of the utmost importance in models for climate studies, weather forecasting, and ocean–atmosphere interaction. The wind stress is mainly obtained by drag coefficient parameterizations, which always consider the wind stress to be aligned with the wind, but this is sometimes the case. Also, during moderate to weak wind conditions, these parameterizations may lead to high estimation errors due to the presence of swell. This study measured the wind stress with a high-rate (100 Hz) sonic anemometer mounted on a spar buoy. The sea state was also characterized by obtaining the directional spectrum of the waves by six wave-staff arrays sensing the free surface level at 10 Hz. Bouy’s movement was corrected by employing an inertial motion unit. The turbulent and wave-coherent wind stress components were also estimated and analyzed. It was observed that during swell conditions with wind traveling in the same direction, the wave-coherent wind stress component has an opposite direction to the wind and dampens the total wind stress magnitude. During counter-directional wind relative to swell events, the wave boundary layer is modified; swell produces a wave-coherent wind stress in the same direction as the wind, resulting in an enhanced total wind stress magnitude. The wave age, significant wave height, and the traveling direction of the swell relative to the wind are essential to correctly estimating the wind stress in swell-dominant conditions. A set of empirical parameterizations for each wind stress component is proposed.