We study cosmological implications of the duality (PSL(2, Z )) invariant potential for the compactification radius T, arising in a class of superstring vacua. We show that in spite of having only one minimum in the fundamental domain of the T field there are two types of non-supersymmetric domain walls: one is associated with the discrete Peccei-Quinn symmetry T→ T+i, analogous to the axionic domain wall, and another one associated with the noncompact symmetry T→ 1 T , analogous to the Z 2 domain walls. The first one is bound by stringy cosmic strings. The scale of such domain walls is governed by the scale of gaugino condensation (O(10 16 GeV) in the case of the hidden E 8 gauge group), while the separation between minima is of the order of M Pl. We discuss the formation of walls and their cosmological implications: the walls must be gotten rid of, either by chopping by stringy cosmic strings and/or inflation. Since there is no Kibble mechanism to create strings, either one must assume they exist ab initio, or one must conclude that string cosmologies require inflation. The non-perturbative potential dealt with here appears not to give the needed inflationary epoch.