The Yellow Sea Cold Water Mass (YSCWM) is an important component of the hydrodynamic system and it significantly impacts the primary production of the Yellow Sea. This study investigated the difference in the interannual variability and long-term trends between the northern YSCWM (NYSCWM) and southern YSCWM (SYSCWM), and explored the main physical environmental factors that led to their inconsistency using multiple wavelet coherence. On the interannual scale, the intensities of the NYSCWM and SYSCWM exhibited consistent variability, but the intensity of the SYSCWM had a larger standard deviation and longer periodic signal than that of the NYSCWM. The two-factor combination of surface air temperature (SAT)–Niño 3.4 in the NYSCWM and sea surface temperature (SST)–northward seawater velocity (Vgos) in the SYSCWM controlled the interannual variability, which meant the influencing intensity variability differed in the NYSCWM and SYSCWM. In the long-term trend, the intensities of the NYSCWM and SYSCWM both showed decreasing trends during the study period. However, the enhanced circulation provided more horizontal heat input into the SYSCWM, and the relatively higher increase in SST and decrease in the amplitude of variation in the thermocline depth promoted vertical heat exchange in the SYSCWM, thereby making the intensity of the SYSCWM decrease more quickly than that of the NYSCWM. These findings provide important references that facilitate a deeper understanding of the influence of hydrological processes on marine ecosystems in marginal seas.