In tropical forest conservation, areas with full statutory protection are often surrounded by buffer zones. Information on the patterns of tree community structure differences in these zones is helpful to evaluate the conservation efficacy. Our study was implemented within a biodiversity hotspot, in the Ta Xua Nature Reserve of north-western Vietnam, which has a statutorily protected core zone and a buffer zone, where local H’Mong people are permitted low intensity forest use. The forests are rich in tree species (249 observed). Many of these tree species provide non-timber forest products (NTFPs) (48%) or valuable timber (22%), and 18 species are red-listed. Overall tree density was not different in the two zones, but tree diameter and species richness were lower in the buffer zone. At the tree level, logistic regression analysis indicated that red-listed status, large diameter, and low density of conspecifics increased the probability of tree absence from the buffer zone but not the potential use as a NTFP. However, most NTFP species had different densities in the core and buffer zones, and this correlated with signs of human interference. At the species level, the density of species was the most important variable, and rarity strongly increased the probability of species absence. Our results also indicate that rare and red-listed trees were depleted in the buffer zone. In consideration of conservation goals, the future monitoring of these species at the Ta Xua Nature Reserve and other protected areas is needed, and conservation measures most likely need to be improved.