An increase in the number of people leaving metropolitan areas (MAs) was observed in various countries in the early years of the COVID-19 pandemic. While considerable attention has been paid to the impacts of health risks and teleworking, two prominent topics related to health-crisis-led migration, empirical evidence remains inadequate. This study aims to empirically investigate the impacts and temporal changes of these two factors on migration leaving MAs (LMA migration). It utilizes survey data from the Japanese government and employs fixed effects logit models. (1) By using infection rates in a more accurate measurement than previous studies, this study confirms the health-risk-aversion motives in LMA migration. (2) Teleworking’s influence on LMA migration is found to be insignificant over the long term. Nevertheless, it increases the likelihood of formal employees staying in MAs and strengthens the tendency of the self-employed to leave for local areas. (3) Temporally, the significant impact of lower COVID-19 infection rates attracting metropolitan residents persisted beyond the pandemic stringency and continued for several months afterward, though it eventually reversed. Teleworking shows a positive influence on LMA migration only in the later stage of COVID-19. These findings suggest a tendency of ‘deferred decisions’ in LMA migration due to people’s unfamiliarity with an unprecedented health crisis. However, the negative impact of infection risks emerges sooner than the significant effect of teleworking, indicating that safety is a pressing priority for LMA migration in the early stages of a major health crisis. (4) Self-employed individuals, homeworkers, and the unemployed are more likely to engage in LMA migration, while employees (whether formal or informal) are less likely, highlighting the role of opportunity costs. Policy implications suggest that local governments should focus on attracting the self-employed from MAs during health crises and on enhancing the teleworking environment for the long term.