Conjugated oligoelectrolytes are of emerging technological interest due to their recent function in the fabrication of optoelectronic devices, application in biosensors, and as species that facilitate transmembrane charge migration. Solubility in aqueous, or highly polar, solvents is important for many of these applications; however, there are few studies on how the self-assembly of conjugated oligoelectrolytes into multichromophore species influences linear and nonlinear optical properties. Here, we examine 1,4-bis(4'-(N,N-bis(6''-(N,N,N-trimethylammonium)hexyl)amino)-styryl)benzene tetraiodide (DSBNI) in water, a conjugated oligoelectrolyte based on the distyrylbenzene framework. We find that DSBNI aggregation leads to increased fluorescence lifetimes, coupled with hypsochromic shifts, and larger two-photon absorption cross sections. Liquid atomic force microscopy (AFM) and cryogenic transmission electron microscopy (cryo-TEM) were used to image DSBNI aggregates and to confirm that the planar molecules stack to form nanocylinders above a critical aggregation concentration. Finally, small-angle neutron scattering (SANS) was used to quantify the aggregate dimensions in situ. Comparison of the results highlights that the hydrophilic mica surface used to image via liquid AFM and the high concentrations required for cryo-TEM facilitate the propagation of the cylinders into long fibers. SANS experiments are consistent with equivalent molecular packing geometry but lower aspect ratios. It is therefore possible to understand the evolution of optical properties as a function of concentration and aggregation and the general geometric features of the resulting supramolecular structures.