Inequality in access to green and blue spaces is a major issue in the field of environmental justice. Various factors significantly influence visitations to these spaces, including residential, commuting, and shopping areas. However, studies have mainly focused on residential environments. Analyzing the coupling of environmental and socioeconomic factors, including age, income, and childhood experience, and examining the direct and indirect effects of these factors for different age groups are needed to better understand the detailed context of inequalities in access to ecosystem services. We conducted a questionnaire survey and analysis to address these needs in an urban setting. The results demonstrate that several land use categories (i.e., agricultural lands in residential areas, grasslands near shopping areas, and deciduous forests near commuting destinations) positively correlated with visitation frequency, suggesting the lack thereof may be the cause of access inequalities. In particular, the perimeter per unit area of some land cover types, which shows the complexity of the form of each land cover category, was found to be a significant factor. We identified inequality in nature experience to be the key factor correlated with inequality in nature visits among the young age group. For the middle-aged and old age groups, nature-relatedness was positively correlated with visitation frequency. Proper design of the local environment, such as through land use planning, can be an effective measure for all age groups. For instance, long-bordered green areas in the Monsoon Asian region are an attractive traditional landscape, with high accessibility and a comfortable thermal environment. The key factors for each age group should be considered in spatial design and inclusive information-sharing to reduce inequality in access to ecosystem services.