Photovoltaic (PV) power generation systems are highly subject to weather and site conditions, thus, the construction of PV power plant projects must consider these uncertainties. This study analyzes the monthly electricity generation of 249 utility-scale PV power plants in Japan to evaluate their electricity generation efficiency. Applying the generic data envelopment analysis, benchmark values were identified for power generation from PV power plants. Furthermore, we implemented a Monte Carlo experiment to evaluate the impact of variability in solar irradiance and temperature on power generation efficiency. For our analysis, we considered three inputs—solar irradiance, temperature, and installed capacity—and electricity generation as the output. The results showed that inter-regional gap in the efficiency score between the west and north regions is 0.03, and this can be covered by a 0.1 increase in the DC/AC ratio. Additionally, variability in weather conditions affect both the efficiency of a power plant and production possibility frontier, in turn causing the benchmark values for a generic decision-making unit to vary. Increasing the generation capacity of power plants and operating them more efficiently is essential to expanding the use of renewable energy resources.