Development of new building HVAC control algorithms has grown due to needs for energy efficiency and operational flexibility. However, case studies demonstrating new algorithms are largely individualized, making algorithm performance difficult to compare directly. In addition, the effort and expertise required to implement case studies in real or simulated buildings limits rapid prototyping potential. Therefore, this paper presents the Building Optimization Testing Framework (BOPTEST) and associated software for simulation-based benchmarking of building HVAC control algorithms. A containerized run-time environment (RTE) enables rapid, repeatable deployment of common building emulators representing different system types. Emulators use Modelica to represent realistic physical dynamics, embed baseline control, and enable overwriting supervisory and local-loop control signals. Finally, a common set of key performance indicators are calculated within the RTE and reported to the user. This paper details the design and implementation of software and demonstrates its usage to benchmark a Model Predictive Control strategy.