Background: The COVID-19 pandemic’s effects on adolescents’ physical activity, sports involvement, and feelings of loneliness remain inadequately understood. This study aimed to explore the shifts in leisure-time physical activity, sports participation, and loneliness among adolescents before and during the pandemic, positing that the pandemic has led to decreased physical activity and sports engagement, as well as heightened loneliness, where more active adolescents experience lower loneliness levels. This study included a prior four-year follow-up cohort from the same region two decades earlier to explore the existence of typical longitudinal aging effects in a cohort not affected by the pandemic. Methods: Prospective and longitudinal data from two cohorts of the Young-HUNT Study two decades apart involving adolescents aged 13–19 years were utilized. The controls were as follows: Cohort 1 from the Young-HUNT1 (YH1) Survey included 2399 adolescents with follow-up in the Young-HUNT2 (YH2) Survey four years later. Cohort 2 included the Young-HUNT4 (YH4) Survey (2017–2019) of 8066 adolescents, with a subset of 1565 participants followed up in the Young-HUNT COVID Survey (YHC) (2021) after exposure to the COVID-19 pandemic and associated restrictions. Changes over time were assessed using McNemar’s tests and dependent sample T-tests, while multinomial logistic regression modeled within-individual changes in loneliness, adjusting for age, gender, and other factors. Results: The findings revealed a significant decline in physical activity and sports participation in both cohorts from early to late adolescence. Additionally, there was a considerable increase in reported loneliness, more after exposure to the pandemic and especially among girls, but without any difference in historical initial levels (between cohorts). Inactive adolescents faced a greater risk of increased loneliness, while those participating in sports had a lower risk of loneliness. Physically inactive boys had a higher risk of loneliness compared with physically active boys at both time points in Cohort 2, which was higher than in the control Cohort 1. There was no historical difference between initial assessments. Conclusion: Adolescents experienced a significant decrease in physical activity and sports participation, along with increased loneliness, from early to late adolescence. Given the protective benefits of physical activity against loneliness and the negative longitudinal trends observed, public health initiatives should focus on increasing physical activity and reducing sports drop-out rates among adolescents to combat rising loneliness.