Composition- and temperature-dependent far-infrared (FIR), terahertz (THz), and low-frequency Raman spectra of poly(3-hydroxybutyrate) (PHB)/Poly(4-vinylphenol) (PVPh) polymer blends were measured to investigate the effects of PVPh in PHB crystallization. FIR, low-frequency Raman, and wide angle X-ray diffraction (WAXD) studies revealed that PVPh reduces the crystallinity of PHB in blends without a significant change in the crystal structure. The FIR and low-frequency Raman spectra divided the blends into three categories: high-ordered crystalline, less-ordered crystalline, and amorphous. A new peak was observed at 135 cm−1 in the FIR spectra of the PHB/PVPh blends, which may be assigned to the less-ordered crystalline phase of PHB, predicting the inter-molecular hydrogen-bond interactions of PHB/PVPh. The intensity ratio of the peaks at 97 and 82 cm−1 changed with the blending ratio variations owing to the crystalline dynamics of PHB. Deformation of the PHB helical structure occurred first, followed by weakening of the intra-molecular hydrogen-bond within PHB. Shifts of several peaks were observed in the FIR and low-frequency Raman spectra, suggesting that the intra-molecular hydrogen-bond (CO⋯H-C-) within PHB weakened with temperature. The novelty of the present study is to demonstrate that low-frequency vibrational spectroscopy is very sensitive to monitor changes from the intra-molecular hydrogen bonding to inter-molecular hydrogen bonding between PHB and PVPh.