Abstract To conserve biodiversity and combat climate change it is vital to restore forest ecosystems. Natural forest regrowth is a nature‐based solution to restore forests, but it has rarely been evaluated how this is affected by the combination of previous land use intensity and surrounding forest cover, and how this varies between the two main tropical forest types; dry and wet forests. Thirty‐three plots were established on abandoned agricultural fields in a dry (13 plots) and wet (20 plots) tropical forest in Mexico and monitored 3 years for the following tree community attributes: structure, diversity, regeneration mode, potential symbioses with N fixing bacteria and mycorrhizal fungi. Previous land use intensity was described using interviews, and landscape forest cover and fragmentation within a 1000 m radius surrounding the plots were quantified using satellite images. Variable importance analyses indicated that land use intensity was more important than forest cover and fragmentation for the state of the tree community attributes after 3 years. This suggests that previous land use impacts the start of succession and leaves important legacies on the vegetation. Land use intensity, forest cover and fragmentation were equally important in determining the change in tree community attributes over time, indicating that both management practices and dispersal shape subsequently community assembly. A higher land use intensity decreases tree richness recovery, while size of the largest forest patch decreases tree density and connectivity of these patches increases tree density. The dry forest had a faster increase in tree density recovery compared to the wet forest through a high initial resprouting capacity and abiotically dispersed trees. Synthesis and applications: Both a higher land use intensity and fragmented forest landscape decreases the speed of forest recovery, indicating that human actions and landscape transformation shape the course of succession. Restoration in fragmented forest landscapes through natural regeneration is more suitable in areas with low previous land use intensity, higher forest connectivity and abiotically dispersed species. Effective restoration should therefore consider both land use history and landscape forest cover and be tailored to local socio‐ecological conditions.