With the increasing number of public buildings worldwide, their energy consumption has garnered significant attention. This study aims to promote building energy efficiency and emission reduction by exploring the application of Building-Integrated Photovoltaic technology in library retrofitting. Using a library in Changsha City as a case study, we conducted an energy consumption analysis of the building’s envelope and identified the window section as the highest energy consumer. We proposed a novel improvement scheme—the sun-shading photovoltaic panel (which shades the sun while generating electricity). Additionally, we utilized the roof space for the installation of conventional photovoltaic panels. Simultaneously, the primary objective of this paper was to derive a renovation strategy for public buildings in regions characterized by high summer temperatures and low winter temperatures based on the renovation case of this library. This paper comprehensively investigates the length and angle of sun-shading photovoltaic panels facing different directions, as well as the azimuth angle, tilt angle, and installation spacing of the conventional photovoltaic panels on the roof. Two retrofitting schemes were proposed. Through simulation and comparison, the second scheme was found to reduce the annual electricity consumption by 1,233,354 kWh, achieving a 35.4% decrease compared to the original building. Using the Changsha library as a template for public buildings, this study provided a retrofitting approach with Building-Integrated Photovoltaic technology via DesignBuilder, which could be extended to other geographic locations. The results demonstrated the significant energy-saving effects of the proposed retrofitting scheme.