Supply voltage overscaling (VOS) has been studied recently for the design of low power finite-impulse response (FIR) filters. The supply voltage is deliberately overscaled for saving power, which introduces errors due to timing violation. These VOS-incurred errors are then compensated by using an estimation-based noise reduction scheme, at the cost of minimal performance degradation while achieving significant power savings. In this paper, novel noise reduction unit (NRU) architectures are proposed to solve two challenging problems. First, we propose to use a separate reliable mechanism for detecting VOS-incurred errors, which frees the estimator from detecting VOS-incurred errors, and consequently enables use of folding techniques and filter adaptation. Second, we propose a novel noise reduction architecture for compensating errors in broadband frequency-selective filters, which show loose statistical dependency that leads to poor estimation in conventional designs. In addition, we recognize that the location of VOS-errors is the most destructive cause to NRU performance degradation, which has not been adequately dealt with in existing literatures. Our proposed architectures are based on smoothing filters, which require minimal hardware overhead while limiting the estimation degradation due to relative dense error distribution. Compared to previous work, the simulation results show that for narrow-band filters our proposed schemes can improve the noise reduction performance by 10-22 dB while achieving the same or better power savings, and for broadband filters the proposed schemes achieve 9-21 dB performance gain while achieving 10.33%-51.74% power savings.