ABSTRACT The residual gain errors add to the systematics of the radio interferometric observations. In case of the high dynamic range observations, these systematic effects dominates over the thermal noise of the observation. In this work, we investigate the effect of time-correlated residual gain errors in the estimation of the power spectrum of the sky brightness distribution in high dynamic range observations. Particularly, we discuss a methodology to estimate the bias in the power spectrum estimator of the redshifted 21-cm signal from neutral hydrogen in the presence of bright extragalactic compact sources. We find, that for the visibility-based power spectrum estimators, particularly those use nearby baseline correlations to avoid noise bias, the bias in the power spectrum arises mainly from the time correlation in the residual gain error. The bias also depends on the baseline distribution for a particular observation. Analytical calculations show that the bias is dominant for certain types of baseline pairs used for the visibility correlation. We perform simulated observation of extragalactic compact sources in the presence of residual gain errors with the Giant Metrewave Radio Telescope like array and estimate the bias in the power spectrum. Our results indicate that in order to estimate the redshifted 21-cm power spectrum, better calibration techniques, and estimator development are required.