The existing literature reveals several significant knowledge gaps that hinder health care providers in formulating exercise prescriptions for cognitive health. This study endeavors to elucidate the relationship between the level of physical activity and cognitive function in older adults in China. Moreover, it seeks to explore the associations between distinct exercise behaviors-such as exercise types, the purpose motivating engagement in exercise, the accessibility of exercise fields, and the inclination toward exercise-and cognitive function. Using data from the China Longitudinal Aging Social Survey (CLASS conducted in 2016, cognitive function was meticulously assessed through the modified Chinese version of the Mini-Mental State Examination, encompassing measures of orientation, memory, and calculation. Using self-report structured questionnaires, a myriad of information about physical activity during leisure time, exercise engagement, exercise intensity, primary exercise types, reasons for exercise participation, availability of sports facilities, and exercise willingness was diligently gathered. Robust ordinary least squares regression models were then used to compute coefficients along with 95% CIs. A discernible inverted U-shaped trend in cognitive scores emerged as the level of physical activity surpassed the threshold of 500 metabolic equivalents of task (MET) minutes per week. Notably, individuals with a physical activity level between 500 and 999 MET minutes per week exhibited a coefficient of 0.31 (95% CI 0.09 to 0.54), those with a physical activity level between 1000 and 1499 MET minutes per week displayed a coefficient of 0.75 (95% CI 0.52 to 0.97), and those with a physical activity level above 1500 MET minutes per week demonstrated a coefficient of 0.45 (95% CI 0.23 to 0.68). Older individuals engaging in exercise at specific MET levels showcased superior cognitive function compared to their inactive counterparts. Furthermore, individuals driven by exercise motivations aimed at enhancing physical fitness and health, as well as those using sports facilities or public spaces for exercise, exhibited notably higher cognitive function scores. The findings underscore the potential of exercise as a targeted intervention for the prevention and treatment of dementia or cognitive decline associated with aging in older individuals. Leveraging these insights to formulate informed exercise recommendations holds promise in addressing a significant public health challenge linked to aging populations.