Traditional credit risk prediction, updates the model from scratch for time-changing data streams, resulting in significant computational, time, and storage consumption. For resource efficiency with acceptable prediction precision, we advocate incrementally training models by transferring knowledge and delayed decisions of low-confidence samples. To this end, we propose a novel multi-stage prediction approach, called Continual Three-Way Decisions, which focuses on knowledge accumulation and thresholds optimization between tasks from the perspective of uncertainty reduction. We utilize elastic weight consolidation to take previous important parameter information as transferred knowledge. Subsequently, we unify continual learning and dynamic three-way decisions through thresholds-based joint mechanism. Finally, the extensive experiments verify the efficacy of our proposed method, demonstrating up to a 38% reduction in running time and a 28% decrease in memory usage compared to isolated and static methods.