The Drinfeld double of a finite dimensional Hopf algebra is a quasi-triangular Hopf algebra with the canonical element as the universal $R$-matrix, and one can obtain a ribbon Hopf algebra by adding the ribbon element. The universal quantum invariant of framed links is constructed using a ribbon Hopf algebra. In that construction, a copy of the universal $R$-matrix is attached to each crossing, and invariance under the Reidemeister III move is shown by the quantum Yang-Baxter equation of the universal $R$-matrix. On the other hand, the Heisenberg double of a finite dimensional Hopf algebra has the canonical element (the $S$-tensor) satisfying the pentagon relation. In this paper we reconstruct the universal quantum invariant using the Heisenberg double, and extend it to an invariant of equivalence classes of colored ideal triangulations of $3$-manifolds up to colored moves. In this construction, a copy of the $S$-tensor is attached to each tetrahedron, and invariance under the colored Pachner $(2,3)$ moves is shown by the pentagon relation of the $S$-tensor.