Examining the relationship between the built environment and older adults’ walking behavior is of critical importance for the development of aging-friendly cities and communities. Previous studies, however, have paid limited attention to the non-linear and synergistic effects of built environment factors. To this end, based on multi-source data such as the Travel Characteristic Survey of Hong Kong and Google Street View imagery, this study integrates two advanced machine learning models—light gradient-boosting machine (LightGBM) and SHapley Additive exPlanations (SHAP)—to analyze the non-linear and synergistic effects of various built environment factors on older adults’ walking time. The results show that the effect of the built environment is largely non-linear. Critical built environment factors include access to recreational facilities and land-use mix. Access to metro and parks, however, plays a marginal role in affecting older adults’ walking. Furthermore, the synergistic effects of built environment variable pairs (e.g., access to recreational facilities and intersection density) are also identified.