The energy performance of commercial buildings is greatly influenced by occupants which are highly variable and among the most unpredictable components of a building's operation. While most building control systems use fixed, predetermined occupancy schedules, these fixed occupancy levels can be quite different from actual occupancy. This can cause unnecessary energy consumption, particularly from heating, ventilation, and air conditioning (HVAC) and lighting systems which are responsible for approximately 60% of commercial buildings' energy use. The use of occupancy counting sensor systems integrated with building management system controls is one method that can be used to improve the energy-consuming performance of buildings. However, there is no standardized universal methodology and metrics to evaluate their reliability. The aim of this research is to develop a uniform evaluation methodology to assess the reliability of occupancy counting sensor systems in a controlled laboratory environment. The developed testing methodology includes both “typical” scenarios representing the occupancy scenarios of a typical commercial building, and “failure” testing scenarios which represent a range of potential scenarios that may impact a sensor system's reliability. These methods were then implemented in a case study to evaluate the performance of two novel occupancy counting sensor systems (i.e., door-centric, and camera-based). Results suggest that typical testing results can be used to compare the overall performance of the occupancy counting sensor systems; however, failure testing is also important to understand the weaknesses of the sensor system in order to select the suitable one for the intended use of the commercial building. In addition, the proposed methodology includes a modified confusion matrix which enables the ability to identify if failures are caused by over or under counting occupants and to what extent this occurs over the testing period.