Soft glasses of colloidal rods (fd-virus particles) with orientational domains were recently shown to exhibit inhomogeneous flow profiles [Dhont et al., Phys. Rev. Fluids 2, 043301 (2017)]: fracture and accompanied plug flow at small shear rates, which transits to gradient shear-banding on increasing the shear rate, while a uniform flow profile develops at sufficiently high shear rates. These flow profiles coexist with Taylor-vorticity bands. The texture of such glasses under flow conditions consists of domains with varying orientations. The observed gradient shear-banding was solely attributed to the strong shear thinning behavior of the material inside the domains (henceforth abbreviated as domain-interior), without considering the texture stress that is due to interactions between the glassy domains. Here, we present new experiments on the shear-banding transition to assess the role played by the texture stress in comparison to the domain-interior stress. For a large concentration, well into the glassy state, it is found that both texture stress and domain-interior stress contribute significantly to the gradient shear-banding transition in the shear-rate region where it occurs. On the other hand, for a small concentration close to the glass-transition concentration, the domains are shown to coalesce within the shear-rate range where gradient shear-banding is observed. As a result, the texture stress diminishes and the domain-interior stress increases upon coalescence, leading to a stress plateau. Thus, a subtle interplay exists between the stresses arising from the structural order on two widely separated length scales from interactions between domains and from the rod-rod interactions within the domain-interior for both concentrations.