Over the past three decades, China has shifted from a relatively immobile society to one where rural migrant workers are dispersed throughout urban areas, resulting in significant changes in rural family structure. Previous studies have tended to approach migrant workers as homogeneous groups within families. In contrast, our attention turns to the diversity among individuals and the complex interactions within families. Based on a survey of rice farmers in five provinces of China, this study aims to explore the heterogeneous impact of changes in rural family structure on the single-factor (i.e., land, labor, and capital) productivity and technical efficiency (TE) of rice production. Methodologically, we calculated the productivity indicator through the Cobb–Douglas production function. Following this, a one-step stochastic frontier approach (SFA) was employed to assess the production frontier and estimate inefficiency. To address self-selection bias in family migration behavior, we applied the propensity score matching method (PSM). The results reveal that significant outcomes are observed only with certain types of changes in rural family structure. The production decisions of rural families are influenced by the migration regions of their family members. Compared to non-migrating families (NM), families with couples’ joint migration outside the province show higher single-factor productivity and TE. We used multiple approaches to examine the results and came to similar conclusions. Therefore, enhancing social security measures and employment opportunities for migrant workers, with specific attention to supporting migrant couples, can have a positive impact on sustainable urban and rural development, as well as food security.