Buildings are by far the largest source of urban energy consumption. In an effort to reduce energy use, cities are mandating that buildings undergo energy benchmarking—the process of measuring building energy performance in order to identify buildings that are inefficient. In this paper, we examine the feasibility of using city-specific, public open data sources in two benchmarking models and compare the results to the same models when using the Commercial Building Energy Consumption Survey (CBECS) dataset, the basis for Energy Star. The two benchmarking models use datasets containing building characteristics and annual energy use from ten major cities. To examine the difference in performance between linear and non-linear models, we use random forest and lasso regression. Results demonstrate that benchmarking models using open data outperform models based solely on the CBECS dataset. Additionally, our results indicate that building area, property type, conditioned area, and water usage are the most important variables for cities to collect. Having demonstrated the benefits of using open data, we recommend two changes to current benchmarking practices: (1) new guidelines that support a data-driven benchmarking framework relying on open data and a transparent modeling process and (2) supporting policies that publicize benchmarking results and incentivize energy savings.