Phonon modes in crystals can have angular momenta in general. It nevertheless cancels in equilibrium when the time-reversal symmetry is preserved. In this Letter, we show that when a temperature gradient is applied and heat current flows in the crystal, the phonon distribution becomes off equilibrium, and a finite angular momentum is generated by the heat current. This mechanism is analogous to the Edelstein effect in electronic systems. This effect requires crystals with sufficiently low crystallographic symmetries, such as polar or chiral crystal structures. Because of the positive charges of the nuclei, this phonon angular momentum induces magnetization. In addition, when the crystal can freely rotate, this generated phonon angular momentum is converted to a rigid-body rotation of the crystal, due to the conservation of the total angular momentum. Furthermore, in metallic crystals, the phonon angular momentum will be partially converted into spin angular momentum of electrons.