Insect herbivory on plant leaves is a major determinant of plant fitness, especially the growth and survival of tree seedlings in forests. Leaf size is believed to significantly affect the intensity of herbivory. Studies often assume the relationship between leaf size and herbivory to be monotonic; however, this relationship is influenced by many factors—the magnitude and direction of which are different—indicating a complex non-monotonic pattern. In this study, we investigated the herbivory of 5754 leaves of 422 seedlings belonging to 42 subtropical tree species over two seasons in southwest China. The effects of leaf size on herbivory differed among seasons; a hump-shaped pattern was detected in December, while a pattern of monotonic increase was detected in September. A variety of patterns, including complex non-monotonic hump-shaped patterns, as well as patterns indicating monotonic decrease and increase existed among species, although most species displayed no significant correlations. The relationship between leaf size and insect herbivory did not follow a constant rule, but differed across species and seasons, indicating that the effects of leaf size on the foraging preferences of insect herbivores may be contingent on both external (e.g., temperature) and intrinsic (e.g., other leaf traits) factors. Therefore, a one-off survey focusing on few species may not provide complete understanding of the overall pattern of the effect of leaf size on herbivory. Similar variations may also exist in other ecological processes, which should be given due consideration in future studies on biotic interactions.