Based on experiments and simulations, various plasma parameters are found to undergo a hysteresis as a function of the driving voltage amplitude in capacitively coupled CF4 discharges. Phase Resolved Optical Emission Spectroscopy reveals that the discharge operates in a hybrid combination of the drift-ambipolar and α-mode at low voltage. In this mode, the electric field and mean electron energy are high in the electronegative plasma bulk region. As the cross section for electron attachment is appreciable only at high electron energies, this mode results in strong negative ion production and keeps the electron density low as well as the mode of plasma operation stable, when the voltage is increased moderately. Increasing the driving voltage amplitude further ultimately induces a mode transition into a pure α-mode, once the electron density increases strongly. Decreasing the voltage again results in a reverse mode transition at a lower voltage compared to the previous mode transition, because the electron density is now initially high in the bulk and, thus, the bulk electric field and mean electron energy are low resulting in inefficient generation of negative ions via electron attachment. This keeps the electron density high even at lower driving voltages. This effect leads to the emergence of two steady states of plasma operation within a certain voltage range. The different electron energy distribution functions in these two states result in markedly different generation and density profiles of F atoms, with higher values occurring in the increasing voltage branch of the hysteresis. The ion flux and mean energy at the electrodes also differ. The voltage range, where the hysteresis occurs, is affected by the ion induced secondary electron coefficient (γ). A larger value of γ results in a shift of the hysteresis voltage range towards lower values.