Physical activity questionnaires (PAQs) are a popular method of monitoring physical activity, although their validity is usually low. Descriptions of physical activity levels in questionnaires usually rely on physical responses to physical activity. Therefore, we hypothesised that the validity of PAQs would be higher in the more physically fit group of participants. To test this, we conducted a validation study with 179 adults whom we divided into three fitness groups based on their cardiovascular fitness and age. Participants were measured for one week using the UKK RM42 accelerometer and self-reported their physical activity using IPAQ-SF, GPAQ, and EHIS-PAQ. We analysed the differences between fitness groups in terms of validity for each PAQ using ANOVA. We also performed an equivalence testing to compare the data obtained with the PAQs and the accelerometers. The results showed a significant trend toward higher validity for moderate to vigorous physical activity from the low to high fitness group as assessed by GPAQ and IPAQ-SF (low, intermediate and high fitness group: 0.06-0.21; 0.26-0.29; 0.40, respectively). The equivalence testing showed that all fitness groups overestimated their physical activity and underestimated their sedentary behaviour, with the high fitness group overestimating their physical activity the least. However, EHIS-PAQ was found to agree best with accelerometer data in assessing moderate to vigorous physical activity, regardless of fitness group, and had a validity greater than 0.4 for all fitness groups. In conclusion, we confirmed that when using PAQs describing physical responses to physical activity, participants' fitness should be considered in the interpretation, especially when comparing results internationally.