It is vital from the public health and educational perspective to be familiar with changes in the fitness levels of individuals and populations since fitness is associated with several health outcomes and cognition. Skill-related physical fitness refers to performance in sports or occupation and is associated with motor skill performance. The aim of the present study was to examine secular trends in skill-related physical fitness of 16 678 participants in four youth generations of Slovenian children and adolescents in years 1983 (n = 3128), 1993/94 (n = 3413), 2003/04 (n = 5497), and 2013/14 (n = 4640). Using repeated cross-sectional design, we observed fitness level of all participants divided into three age groups: 6-10, 11-14, and 15-19 years. Skill-related physical performance was measured with seven fitness tests for speed, coordination, balance, and flexibility. Analysis of covariance was used to compare differences in fitness performance between decades in each age and sex group, adjusted for body height, body weight, and body mass index. Overall, large but inconsistent changes in coordination, a small improvement in speed, and a decline in flexibility were seen. The trends over the whole examined period were not linear throughout decades. Generally, positive trends were noticed in periods 1983-1993 (range 1.4%-17.9%; except flexibility) and 2003/04-2013/14 (range 0.2%-36.4%; except age group 15-19 years) while in the period 1993/94-2003/04 there are some particularities in secular trends according to individual components as well as age groups. In general, the secular trend showed a positive direction for both genders (p < 0.05), except for gross motor coordination, which demonstrated positive trends in 1993 and 2013 compared with a decade earlier (p < 0.05) and from 1983 to 2013/14, except for the youngest boys in 2003 and the youngest girls from 1993 to 2003 (p < 0.05). Our findings call for exercise programs aimed at improving speed and gross motor coordination in both sexes and all age groups, especially in the group of 15-19 years old.