Non-contact sensors are increasingly integral to the field of Structural Health Monitoring (SHM), with camera being widely utilized due to their high spatial resolution and cost-effectiveness in measuring structural displacements. Phase-based optical flow (POF) techniques have proven valuable for accurate and computationally efficient 2D vibration measurements. However, these techniques are typically confined to single-camera systems and encounter challenges in measuring vibrations in out-of-plane directions. Previous research on 3D vibration measurement has predominantly centered on pixel intensity-based techniques using stereo camera systems. Although these methods demonstrate high performance in calculating global strains and stresses, they are less suitable for accurate vibration measurement. To address these limitations, this study proposes a novel 3D vibration measurement approach that integrates POF within the context of stereo camera systems. 2D vibrations are extracted by applying Complex Gabor filters to stereo images, and subsequently, 3D vibration measurements are estimated through camera calibration and triangulation, using computed camera internal and external parameters. To validate the method, modal analysis experiments are conducted on an aluminum plate. The 3D vibration measurement results are compared with those obtained through 3D Digital Image Correlation (DIC) and contact sensors and the accuracy is demonstrated.