Multimodal Large Language Models as Built Environment Auditing Tools

Abstract

This research showcases the transformative potential of large language models (LLMs) for built environment auditing from street-view images. By empirically testing the performances of two multimodal LLMs, ChatGPT and Gemini, we confirmed that LLM-based audits strongly agree with virtual audits processed by a conventional deep learning-based method (DeepLabv3+), which has been widely adopted by existing studies on urban visual analytics. Unlike conventional field or virtual audits that require labor-intensive manual inspection or technical expertise to run computer vision algorithms, our results show that LLMs can offer an intuitive tool despite the user’s level of technical proficiency. This would allow a broader range of policy and planning stakeholders to employ LLM-based built environment auditing instruments for smart urban infrastructure management.