β-BiB3O6 was compressed in a diamond anvil cell at room temperature and studied using a combinstion of in situ high-pressure Raman scattering and angle-dispersive synchrotron X-ray diffraction. The results reveal that β-BiB3O6 retains its structure below 9.0 GPa and undergoes a structural phase transition that starts at ~11.5 GPa and finishes at ~18.5 GPa. No other phase transition occurred with increasing pressure up to ~46 GPa. It was also found that the high-pressure phase is different from the already-reported five polymorphs of BiB3O6. Therefore, the new phase is denoted as ζ-BiB3O6, which could be indexed by an orthorhombic unit cell (a ≈ 12.5 Å, b ≈ 6.7 Å, c ≈ 4.0 Å) from the powder XRD pattern collected at 22.2 GPa. Moreover, ζ-BiB3O6 can be quenched to ambient conditions. The investigation of the pressure dependence of the lattice parameters reveals that both β-BiB3O6 and ζ-BiB3O6 exhibit a large amount of crystallographic anisotropy. An unusual expansion of the c-axis of ζ-BiB3O6 was observed. Assignments for the Raman spectra of β-BiB3O6, γ-BiB3O6, and δ-BiB3O6 under ambient conditions were also performed. Currently, we cannot solve the crystal structure of ζ-BiB3O6 but give some speculations based on its relationship with β-BiB3O6.