Developing specialties in orchard fruits productions with ecological and economic benefits is a practical and effective way to guarantee eco-friendliness and increase farmers’ income in the Loess Plateau area. Therefore, to understand these factors, the study constructs an agriculture ecological cognition index from three dimensions of eco-agriculture cognitions (increase income cognition, water conservation cognition and eco-product price cognition). Our analysis was based on micro survey data from 416 farmers in Shaanxi and Ningxia, China. The study used two main econometric models, double-hurdle and Interpretative Structural Modeling (ISM), to examine the relationship and influence pathways between cognition of ecological agriculture and farmers’ specialty orchard fruit planting behavior. The results show that: (i) the cognition of eco-agriculture affects whether farmers plant specialty fruits (participation decision). The cognition of eco-agriculture increases income and the cognition of eco-product price significantly affect the scale of specialty orchard fruits planting (quantity decision). (ii) Household resource endowments influence specialty orchard fruit planting responses through ecological farming cognitions. (iii) The factors influencing the participation and quantity decisions of orchard fruit planting are significantly different. Therefore, when the government actively encourages farmers to participate in specialty orchard planting, it should fully consider the cognitive factors of ecological agriculture of the growers and develop targeted training strategies.