Pseudomonas aeruginosa has long been known to produce phenotypic variants during chronic mucosal surface infections. These variants are thought to be generated to ensure bacterial survival against the diverse challenges in the mucosal environment. Studies have begun to elucidate the mechanisms by which these variants emerge in vitro; however, too little information exists on phenotypic variation in vivo to draw any links between variants generated in vitro and in vivo. Consequently, in this study, the P. aeruginosa gacS gene, which has previously been linked to the generation of small colony variants (SCVs) in vitro, was studied in an in vivo mucosal surface infection model. More specifically, the rat prostate served as a model mucosal surface to test for the appearance of SCVs in vivo following infections with P. aeruginosa gacS(-) strains. As in in vitro studies, deletion of the gacS gene led to SCV production in vivo. The appearance of these in vivo SCVs was important for the sustainability of a chronic infection. In the subset of rats in which P. aeruginosa gacS(-) did not convert to SCVs, clearance of the bacteria took place and healing of the tissue ensued. When comparing the SCVs that arose at the mucosal surface (MS-SCVs) with in vitro SCVs (IV-SCVs) from the same gacS(-) parent, some differences between the phenotypic variants were observed. Whereas both MS-SCVs and IV-SCVs formed dense biofilms, MS-SCVs exhibited a less diverse resistance profile to antimicrobial agents than IV-SCVs. Additionally, MS-SCVs were better suited to initiate an infection in the rat model than IV-SCVs. Together, these observations suggest that phenotypic variation in vivo can be important for maintenance of infection, and that in vivo variants may differ from in vitro variants generated from the same genetic parent.