The severity of infectious disease outbreaks is governed by patterns of human contact, which vary by geography, social organization, mobility, access to technology and healthcare, economic development, and culture. Whereas globalized societies and urban centers exhibit characteristics that can heighten vulnerability to pandemics, small-scale subsistence societies occupying remote, rural areas may be buffered. Accordingly, voluntary collective isolation has been proposed as one strategy to mitigate the impacts of COVID-19 and other pandemics on small-scale Indigenous populations with minimal access to healthcare infrastructure. To assess the vulnerability of such populations and the viability of interventions such as voluntary collective isolation, we simulate and analyze the dynamics of SARS-CoV-2 infection among Amazonian forager-horticulturalists in Bolivia using a stochastic network metapopulation model parameterized with high-resolution empirical data on population structure, mobility, and contact networks. Our model suggests that relative isolation offers little protection at the population level (expected approximately 80% cumulative incidence), and more remote communities are not conferred protection via greater distance from outside sources of infection, due to common features of small-scale societies that promote rapid disease transmission such as high rates of travel and dense social networks. Neighborhood density, central household location in villages, and household size greatly increase the individual risk of infection. Simulated interventions further demonstrate that without implausibly high levels of centralized control, collective isolation is unlikely to be effective, especially if it is difficult to restrict visitation between communities as well as travel to outside areas. Finally, comparison of model results to empirical COVID-19 outcomes measured via seroassay suggest that our theoretical model is successful at predicting outbreak severity at both the population and community levels. Taken together, these findings suggest that the social organization and relative isolation from urban centers of many rural Indigenous communities offer little protection from pandemics and that standard control measures, including vaccination, are required to counteract effects of tight-knit social structures characteristic of small-scale populations.