Climate change leads to more frequent and intense heat waves, exposing urban populations to extreme heat conditions and significant health risks. Many cities are adopting Nature-based Solutions (NbS) to mitigate urban heat, with green roofs emerging as a widely recognized NbS. Although numerous green roof projects are implemented on a small scale, research on the effectiveness of small-scale green roofs on an urban scale is limited, making it challenging to identify the critical factors for both maladaptation and success. Therefore, we assessed the cooling potential of 18 small-scale green roofs in Seoul, Republic of Korea. This comparative study that adopted a multi-site approach examined the cooling capacity of green roofs to reduce surface temperature and identified characteristics of effective and ineffective green roofs. We utilized a developed difference-in-differences method to improve causal inference, effectively isolating the effects of individual green roofs from background climate change. The multi-site comparative approach and more robust causal inference methods improved our understanding of the effects of small-scale green roofs. The findings of this study indicate that three out of 18 green roofs were statistically significant maladaptation cases with an increase in LST. This evidence can help an urban planner reduce ineffectiveness and enhance effective adaptation practices. Our proposed method is expected to support government projects, especially those with limited budgets, in efficiently managing urban heat and reducing trial and error.