Physical literacy (PL) has received considerable attention in the field of physical education and physical activity (PA) worldwide. According to recent studies, the level of physical fitness (PF) among Chinese university students is gradually decreasing. This study aims to examine the impact of the PF level (fit/unfit) on PA and PL, as well as the relationships among PF, PA, and PL, in Chinese university students. Participants comprised 798 university students (390 men; mean age, 19.2 ± 1.2 years) in Chongqing, China. Participants completed the tests of vital capacity, cardiorespiratory fitness, muscular strength, and flexibility, according to the National Physical Fitness Measurement Standards Manual (NPFMSM), as well as questionnaires on PA (time spent performing PA at various intensities) and PL. The independent t-tests were conducted to examine sex differences in the evaluated variables, and the Pearson’s correlation between each PF test and PL attributes and PA was calculated according to sex. In addition, the independent t-tests were conducted to determine whether the PF level had an effect on PL attributes and PA at various intensities. Significant sex differences were found in the PF domains of vital capacity, muscular strength, and aerobic fitness, with higher scores in men than in women (all p-values < 0.05), but not in the PF domain of flexibility and total PL score. Furthermore, the PF domains of muscular strength and aerobic fitness were significantly and positively correlated with the PL attributes of confidence and physical competence in both men and women, while the PF domains of vital capacity and aerobic fitness were significantly and positively correlated with the PL attribute of motivation in men. In addition, PL was significantly and positively associated with cardiorespiratory fitness, vital capacity, muscular strength, and flexibility among participants in the fit group. These findings support advocating for increased participation in PA in university students and using PL as a tool to improve PF components.