With the increase of density and complexity of high performance integrated circuits and systems, including many-core chips and system-on-chip (SoC), it is becoming difficult to meet the power delivery and regulation requirements with off-chip regulators. The off-chip regulators become a less attractive choice because of the higher overheads and complexity imposed by the additional wires, pins, and pads. The increased $I^{2}R$ loss makes it challenging to maintain the integrity of different voltage domains under a lower supply voltage environment in the smaller technology nodes. Fully integrated on-chip voltage regulators have proven to be an effective solution to mitigate power delivery and integrity issues. Recently, there has been a surge of interest among the academic and industrial research communities to explore and design different types of on-chip voltage regulators. This survey presents a brief overview of the on-chip power delivery system and the classification and working principles of on-chip voltage regulators. The main focus of this review paper is to provide a comprehensive study of two of the most promising on-chip voltage regulators – (i) the low-drop-out (LDO) regulator and (ii) the switched-capacitor (SC) regulator. This article elaborates on the design specifications, optimization techniques, advantages, and limitations of LDO and SC type on-chip voltage regulators.