Sleep and depression are strongly associated with cognitive impairment. The role of sleep disturbances in the adverse effect of depression on cognitive dysfunction in older adults remains unclear. This study explored the mediating effect of self-reported sleep disturbances on the relationship between depression and cognitive impairment in older adults according to sex differences. This study derived data from the 2009 Taiwan National Health Interview Survey and included 2,175 community-dwelling adults aged 65 years and older (men = 991; women = 1,184). Sleep disturbances were measured using self-reported survey questions. The Center for Epidemiological Studies Depression scale was used to assess depression. The Mini-Mental State Examination was used to evaluate cognitive impairment. A higher proportion of female older persons had cognitive impairment and depression than male older persons (cognition: 24.4% vs. 11.5%; depression: 17.0% vs. 10.8%). The meditating effect of sleep was detected in only men. Difficulty in initiating sleep was a complete mediator of the adverse effect of depression on cognitive impairment (Sobel test: p = .03). In summary, difficultly in initiating sleep may be a crucial, treatable mediator of the adverse effect of depression on cognitive impairment in older men.