It has been shown that optimizing quadratic costs while stabilizing affine control systems to desired (sets of) states subject to state and control constraints can be reduced to a sequence of quadratic programs (QPs) by using control barrier functions (CBFs) and control Lyapunov functions (CLFs). In this article, we introduce adaptive CBFs (aCBFs) that can accommodate time-varying control bounds and noise in the system dynamics while also guaranteeing the feasibility of the QPs if the original quadratic cost optimization problem itself is feasible, which is a challenging problem in current approaches. We propose two different types of aCBFs: parameter-adaptive CBF (PACBF) and relaxation-adaptive CBF (RACBF). Central to aCBFs is the introduction of appropriate time-varying functions to modify the definition of a common CBF. These time-varying functions are treated as high-order CBFs with their own auxiliary dynamics, which are stabilized by CLFs. We demonstrate the advantages of using aCBFs over the existing CBF techniques by applying both the PACBF-based method and the RACBF-based method to a cruise control problem with time-varying road conditions and noise in the system dynamics, and compare their relative performance.