Topological insulators have been the subject of intense research interest due to their unique surface states that are topologically protected against scattering or defects. However, the relationship between the crystal structure and topological insulator state remains to be clarified. Here, we show the effects of hydrostatic pressure on the structural, vibrational, and topological properties of the topological insulator Bi1.5Sb0.5Te1.8Se1.2 up to 45 GPa using X-ray diffraction and Raman spectroscopy in a diamond anvil cell, together with first-principles theoretical calculations. Two pressure-induced structural phase transitions were observed: from ambient rhombohedral R3¯m phase to a monoclinic C2/m phase at ∼13 GPa, and to a disordered I4/mmm phase at ∼22 GPa. In addition, the alloy undergoes several electronic transitions within the R3¯m phase: indirect to direct bulk band gap transition at ∼5.8 GPa, bulk gap closing with an appearance of Dirac semimetal (DSM) state at ∼8.2 GPa, and to a trivial semimetal state at ∼12.1 GPa. Anomalies in c/a ratio and Raman full width at half maximum that coincide with the DSM phase suggest the contribution of electron-phonon coupling to the transition. Compared to binary end members Bi2Te3, Bi2Se3, and Sb2Te3, the structural phase transition and anomaly were observed at higher pressures in Bi1.5Sb0.5Te1.8Se1.2. These results suggest that the topological transitions are precursors to the structural phase transitions.