Using steel slag to replace natural coarse and/or fine aggregates to make concrete provides a multi-win solution in mitigating the environmental problems regarding huge depletion of natural aggregates and disposal of large amount of slag. On the other hand, the use of pozzolanic fillers, such as fly ash (FA) and silica fume (SF) could significantly improve the strength, stiffness and durability of concrete. However, there were very few research studies on the effect of pozzolanic fillers on the mechanical behaviour and durability of concrete containing steel slag. To fill up this research gap and promote the use of steel slag, a systematic and scientific experimental study was conducted, in which 29 concrete mixes with steel slag partially or totally replacing natural aggregates and FA/SF partially replacing cement in an equal volume were prepared. The slump, wet packing density (WPD), 7-, 28-, 91- and 182-day compressive strength and durability in terms of total charge passed and chloride ion’s penetration depth in rapid chloride permeability test were tested for all the specimens while the chemical composition of concrete were tested using X-ray diffraction. Test results revealed that the slump of steel slag concrete decreased due to the porous structure of slag. Besides, the use of steel slag improved the compressive strength of concrete. By adding SF to replace partial cement, the strength of steel slag concrete could be escalated further. At given composition of aggregates and filler type, the total charge passed and the chloride ion’s penetration depth of steel slag concrete decreased as the WPD increased due to the denser and impermeable microstructure of concrete that enhanced the resistance of chloride ion penetrability.