Tree diversity and landscape features have long been considered as the main drivers of insect herbivory in forest ecosystems. However, relatively little is known about how local and landscape factors, directly and indirectly, acting at different spatial scales affect insect damage under natural, non-outbreak conditions. Here, 30 mixed and pure forest stands were selected in natural forests along the Irtysh river in Xinjiang, Northwest China. To assess the effects of landscape-scale forest isolation, tree diversity, stand position, tree size, understory vegetation cover, and leaf traits on leaf herbivory on Populus laurifolia, we measured three leaf traits (i.e., leaf area, specific leaf area, and leaf dry matter content) and defoliation intensity of insect herbivores in host poplars in 30 sampling plots within mixed and pure forest stands. We also explored the indirect effects of tree diversity on insect herbivory via changes in leaf traits. We found that both forest isolation and tree height had positive effects on insect herbivory, but vegetation cover negatively affected it. Insect herbivory was lower in mixed than in pure forest stands, however, this difference was only significant in stands at the near distance from the river. All leaf traits differed significantly between mixed and pure stands, and varied with tree height. In addition, insect herbivory was positively correlated with leaf area and specific leaf area. However, the influence of tree diversity and tree size on insect herbivory mainly act indirectly through changes in leaf area and specific leaf area. Our results indicate that insect herbivory can be driven by a complex system of ecological factors at multiple scales, and offer an insight into the key role of leaf traits in driving tree diversity-herbivory interactions in a natural forest ecosystem. Thus, these findings are crucial to improve risk assessment and potential prevention strategies of insect damage in riparian forests in real-world situations.