Silica fume is usually used in UHPC, three times more than that for normal concrete, to enhance mechanical properties and durability. However, silica fume (SF) is an expensive material and has high production costs. This work is aimed at investigating the shrinkage and durability performance of previously developed UHPC mixtures utilizing the two calcareous waste materials, namely limestone powder (LSP) and cement kiln dust (CKD), by partially replacing the silica fume. The optimally selected mixtures of UHPC, having flow and strength above the minimum required, were used for detailed investigation in terms of shrinkage and durability characteristics. The results showed that by replacing SF with up to 20% of LSP and up to 20% of CKD, the mechanical properties of UHPC remained satisfactory compared to the control mixture with 100% SF. However, the ultimate shrinkage was higher for mixtures incorporating LSP or CKD, indicating the need for more volume of steel fibers to compensate for the shrinkage strains. The developed UHPCs also exhibited high resistance against reinforcement corrosion and sulfate attack, making them suitable for use in aggressive exposure conditions. However, special attention needs to be paid to the CKD content, where it is recommended to limit the content of CKD to about 15% or less to control the durability performance of the UHPCs. In addition, the sustainability analysis of developed UHPC mixtures was carried out using the life-cycle assessment and eco-strength intensity index. The results indicated that the UHPC mixtures possess a higher life-cycle and are therefore more sustainable.