Cognitive frailty is the coexistence of physical frailty and mild cognitive impairment. Research shows that cognitive frailty is related to an increased risk of hospitalization, mortality, disability, and dementia. Diabetes and hypertension are common risk factors for physical frailty and cognitive impairment. However, the factors influencing cognitive frailty in the elderly with hypertension and diabetes are still unclear. This study aimed to investigate the possible factors influencing cognitive frailty in the elderly with hypertension and diabetes. A cross-sectional study was conducted. We evaluated people over 60 years with hypertension and diabetes who underwent physical examination in Wuxi Xin'an Community Health Service Center. Frail scale, Montreal Cognitive Assessment-Basic and clinical dementia rating were used to assess cognitive frailty. We collected demographic characteristics, hypertension and diabetes-related laboratory indicators of the participants. We also used various scales to assess the overall health status of the elderly. Approximately 20.8% of the participants were determined to have cognitive frailty in elderly adults with hypertension and diabetes. These participants were older, had a lower monthly income, and included a higher proportion of peasants. They also had a higher level of depression (p = 0.037), higher risk of falls (p = 0.000), higher risk of malnutrition (p = 0.002), poorer ability to perform activities of daily living (ADL) (p = 0.000), and less social support (p = 0.030). Multivariate regression analysis was used to further assess the factors for cognitive frailty. After adjusting for possible confounders, age and ADL score emerged as risk factors, whereas high monthly income decreased the risk of cognitive frailty. Cognitive frailty is correlated with age, income, and ability to perform daily living activities in the elderly with diabetes and hypertension. Closer attention to the elderly who have low income and poor self-care ability may play an important role in the early prevention of cognitive frailty and even dementia.