Denote by $\widetilde{\mathrm{U}}(p,q)$ the universal covering group of $\mathrm{U}(p,q)$, the linear group of isometries of the pseudo-Hermitian space $\mathbb{C}^{p,q}$ of signature $p,q$. Let $M$ be a connected analytic complete pseudo-Riemannian manifold that admits an isometric $\widetilde{\mathrm{U}}(p,q)$-action and that satisfies $\dim M \leq n(n+2)$ where $n = p+q$. We prove that if the action of $\widetilde{\mathrm{SU}}(p,q)$ (the connected derived group of $\widetilde{\mathrm{U}}(p,q)$) has a dense orbit and the center of $\widetilde{\mathrm{U}}(p,q)$ acts non-trivially, then $M$ is an isometric quotient of manifolds involving simple Lie groups with bi-invariant metrics. Furthermore, the $\widetilde{\mathrm{U}}(p,q)$-action is lifted to $\widetilde{M}$ to natural actions on the groups involved. As a particular case, we prove that when $\widetilde{M}$ is not a pseudo-Riemannian product, then its geometry and $\widetilde{\mathrm{U}}(p,q)$-action are obtained from one of the symmetric pairs $(\mathfrak{su}(p,q+1), \mathfrak{u}(p,q))$ or $(\mathfrak{su}(p+1,q), \mathfrak{u}(p,q))$.