Novel hybrid beamformer designs are conceived for a multi-user multi-cell (MUMC) mmWave system relying on base station (BS) coordination and total transmit power minimization subject to realistic signal-to-interference-plus-noise ratio (SINR) constraints at each mobile station (MS). Initially, a semidefinite relaxation (SDR)-based approach is developed for a centralized MUMC system to determine the fully digital beamformer having perfect CSI. Subsequently, a Bayesian learning (BL) technique is harnessed for decomposing the fully-digital (FD) solution into its analog and digital components for constructing a hybrid transceiver. Next, an alternating direction method of multipliers (ADMM) based distributed hybrid beamformer is designed for the same system, which requires only local CSI and limited information exchange among the BSs, thus avoiding the excessive signalling overheads required by the centralized approach. Then we further extend both the centralized and the above distributed hybrid designs to construct robust beamformers that minimize the worst-case transmit power with imperfect CSI. Our robust beamforming techniques leverage the S-lemma, which is eminently suitable for the infinitely many constraints arising from the associated CSI uncertainty. Finally, our simulation results demonstrate the improved performance of the proposed centralized and distributed methods over the system having no coordination.