As a type of macronutrient, fatty acids (FA) play significant roles in the bone health of elderly people. However, the specific association between different types of FA and bone health is not fully understood, especially in rural elderly populations. To address this gap, a study was conducted in rural areas of Qingdao, China. Participants aged 65 and older were randomly recruited from 11 rural villages in Licha town, Qingdao City. The levels of serum FA in their serum were measured to investigate the associations between FA and bone mass. The results showed that levels of saturated fatty acids (SFA), n-3 polyunsaturated fatty acids (n-3 PUFA), and n-6 polyunsaturated fatty acids (n-6 PUFA) were all significantly associated with bone mass. Specifically, higher levels of SFA were positively correlated with low bone mass (LBM), while PUFA levels were inversely correlated with LBM. Furthermore, the odds ratio (OR) for LBM exhibited a significant nonlinear dose-response relationship (pnonlinearity = 0.1989) with SFA levels, and a significant nonlinear dose-dependent relationship was also observed with the levels of n-3PUFA and n-6PUFA (pnonlinearity = 0.6183, 0.5808, respectively), indicating that increasing dietary PUFA intake appropriately and controlling SFA intake may benefit the bone health of elderly individuals in rural areas.