While research on building-integrated photovoltaics (BIPVs) has mainly focused on power-generating window applications, the utilization of other underutilized surface areas in buildings, including exteriors, facades, and rooftops, has still not been fully explored. The most important requirements for BIPVs are color, power conversion efficiency (PCE), and long-term stability. In this work, we achieved colorful (red, green, blue, RGB) perovskite solar cells (PSCs) with minimized PCE loss (<10%) and enhanced photostability by exploiting the optical properties of nonperiodic multi-nanolayer, narrow-bandwidth reflective filters (NBRFs). The NBRFs were fabricated by multilayering high-index TiO2/low-index SiO2 in a nonperiodic manner, which allowed devices to demonstrate various colors with effectively suppressed unwanted baseline ripple-shape reflectance. The PCEs of PSCs with nonperiodic RGB-NBRFs were 18.0%, 18.6%, and 18.9%, which represent reductions of only 10%, 7%, and 6% of PCE values, respectively, compared to a black control PSC (20.1%). Moreover, the photostability of the PSCs was substantially improved by using the NBRFs because of ultraviolet blocking in the TiO2 layers. The G-PSC retained 65% of the initial PCE after 60 h of continuous illumination (AM 1.5G one sun) at the maximum power point, whereas the black PSC retained only 30%. Aesthetic color value, low PCE loss, and enhanced photostability of PSCs were simultaneously achieved by employing our NBRFs, making this a promising strategy with potential applicability in power-generating building exteriors.