Creating statistically equivalent virtual microstructures (SEVM) for polycrystalline materials with complex microstructures that encompass multi-modal morphological and crystallographic distributions is a challenging enterprise. Cold spray-formed (CSF) AA7050 alloy containing coarse-grained prior particles and ultra-fine grains (UFG) and additively manufactured (AM) Ti64 alloys with alpha laths in beta substrates. The paper introduces an approach strategically integrating a Generative Adversarial Network (GAN) for multi-modal microstructures with a synthetic microstructure builder DREAM.3D for packing grains conforming to statistics in electron backscatter diffraction (EBSD) maps for generating SEVMs of CSF and AM alloy microstructures. A robust multiscale model is subsequently developed for self-consistent coupling of crystal plasticity finite element model (CPFEM) for coarse-grained crystals with an upscaled constitutive model for UFGs. Sub-volume elements are simulated for efficient computations and their responses are averaged for overall stress-strain response. The methods developed are important for image-based micromechanical modeling that is necessary for microstructure-property relations.