Amidst the growing concerns about mental health issues among urban dwellers, there has been increased attention to the restorative potential (RP) of urban landscapes. However, research into the relevant landscape features tends to lack the integration of quantifiable methodologies, which are essential for practical implementation in urban planning and management. To address this gap, our study employed panorama-based metrics relying on computer vision to quantify the features of urban landscapes. We established an explanatory (generalized linear mixed model, GLMM) and a predictive (Random Forest) model to comprehensively elucidate the relationship between these landscape features and RP. The RP outcome was derived from an online survey of respondents (n = 1500) with diverse socio-demographic characteristics, where each respondent was randomly assigned panoramic images from a pool of a hundred landscape scenes taken across the city of Singapore. The GLMM regression identified sky, tree, water, and depth as the primary influences of RP. Furthermore, the Random Forest model demonstrated commendable predictive performance and was used to visually showcase the scenes within the testing dataset with corresponding RP outcomes. To illustrate the potential application of this predictive tool, we also performed spatial mapping for RP prediction, taking the Singapore Botanic Gardens as an example. The subsequent discussion underscored the simultaneous effect of tree, sky, and depth on RP from a quantitative perspective using LOWESS curves. By employing metrics and models that explain and predict the RP, this study contributes evidence-based insights that underscore the pivotal role of trees and openness (sky and depth) in enhancing the psychological benefits of urban landscapes and provides an effective predictive tool that contributes to urban landscape planning and management.