Starting from microscopic models of space-time foam, based on braneuniverses propagating in bulk space-times populated by D0-branedefects (``D-particles''), we arrive at effective actions used by alow-energy observer on the brane world to describe his/herobservations of the Universe. These actions include, apart from themetric tensor field, also scalar (dilaton) and vector fields, thelatter describing the interactions of low-energy matter on the braneworld with the recoiling point-like space-time defect(D-particle). The vector field is proportional to the recoil velocityof the D-particle and as such it satisfies a certain constraint. Thevector breaks locally Lorentz invariance, which however is assumed tobe conserved on average in a space-time foam situation, involving theinteraction of matter with populations of D-particle defects. In thispaper we clarify the role of fluctuations of the vector field onstructure formation and galactic growth. In particular we demonstratethat, already at the end of the radiation era, the (constrained)vector field associated with the recoil of the defects provides theseeds for a growing mode in the evolution of the Universe. Such agrowing mode survives during the matter dominated era, provided thevariance of the D-particle recoil velocities on the brane is largerthan a critical value. We note that in this model, as a result ofspecific properties of D-brane dynamics in the bulk, there is no issueof overclosing the brane Universe for large defect densities. Thus, inthese models, the presence of defects may be associated withlarge-structure formation. Although our string inspired models dohave (conventional, from a particle physics point of view) dark mattercomponents, nevertheless it is interesting that the role of``extra'' dark matter is also provided by the population of massivedefects. This is consistent with the weakly interacting character ofthe D-particle defects, which predominantly interact onlygravitationally.