The field of urban building energy modeling, embracing diverse aspects such as geography, construction, and materials. There is a lack of a comprehensive information integration framework to streamline cross-domain data in a systematic manner and generate simulation files that are easily calculable. This study addresses this gap by proposing two key ontologies: Building Template Ontology for managing building energy simulation templates and Urban Building Ontology for organizing building physics information. Utilizing this ontology-driven method, this study enhances flexibility in information fusion, exploits logical relationships between simulation inputs, and provides a lightweight solution for structured urban energy performance analysis. Energy simulations were conducted on over 5,000 buildings across three cities, and the impact of energy retrofit measures was further quantified, revealing potential savings of 0.1 % to 4.0 %, 1.2 % to 10.2 %, and 2.5 % to 6.9 % for building envelope, lighting, and air conditioning improvements, respectively. This study empowers urban stakeholders, designers, and managers to streamline the model construction and provides valuable insights into energy consumption patterns. Acknowledging limitations in current ontologies, including restricted building type templates, lack of real-time instance information, and absence of direct energy verification, this study underscores the imperative to address these for future advancements.