Plenty of previous studies have observed the cross-modal correspondence effect of vision and taste. In particular, affective pictures were shown to be associated with certain taste words. However, the extent to which such associations influence real taste perception remains unclear. The current study aims to assess the role of affective visual stimuli in the cross-modal modulation of sweet taste sensitivity. We utilized the same affective pictures from our previous study and categorized them into nine sub-groups based on different valence and arousal levels. Twenty volunteers (7 males and 13 females, average age 21.0 ± 0.7 years) were recruited to participate in the experiment. They were asked to taste various sucrose solutions after viewing each affective picture. Results showed that affective pictures' different valence and arousal levels could modify sweetness sensitivity. Specifically, sweetness sensitivity increases as the valence level changes from negative to positive. Positive pictures with high valence and intense arousal had the highest sweet sensitivity. On the other hand, as the arousal levels of negative pictures increased, sweetness sensitivity gradually decreased. Moreover, this visual-gustatory cross-modal integration was most significant around the sweetness threshold level. This study extends our understanding of the cross-modal interaction between vision and taste in a systematically tailored pattern. Specifically, affective visual stimuli have been shown to influence sweet taste sensitivity, with both valence and arousal levels playing essential roles. Such findings provide new insights into the complex interplay between sensory modalities and emotional states in shaping our taste perception.