Abstract Longitudinal studies have identified an increase in psychological distress across the general population during the COVID-19 pandemic. Nevertheless, the trajectories of mental health outcomes exhibit variations, suggesting potential associations with psychosocial individual factors. This paper identifies the factors of this individual trajectory of psychological distress during the COVID-19 pandemic. Five waves of a prospective cohort survey were conducted with a convenience sample of the general population in Belgium between March 2020 and November 2021 (n = 4,550). Psychological distress was measured using the GHQ-12. Individual covariates included socioeconomic factors (age, gender, level of education), psychological factors (loneliness, social support, and social activities), and factors related to the virus and the lockdown measures (exposure to COVID-19 and survey wave). Multilevel models were used for analysis. Women and young people experienced more pronounced fluctuations in their risk of psychological distress across study waves, experiencing both increases and decreases. We found that individual variance in psychological distress breaks down into two components, respectively 43% for psychosocial factors (time-invariant) and 57% for the survey waves (time-variant) variation. A significant share of the time-invariant difference in psychological distress over COVID-19 is associated with loneliness, social support, and social activities. Loneliness emerged as the most important interpersonal factor associated with psychological distress. The change in psychological distress was mainly associated with psychosocial factors rather than with pandemic-related dynamics (e.g. survey waves). These findings suggest that mitigation policies aiming at controlling the pandemic should focus more on addressing specific individual vulnerabilities rather than solely responding to the fluctuations within pandemic waves to decrease their detrimental impact on mental health.