Quarry sludge (QS) is a common by-product of quarry mining and the subsequent aggregate washing process, but its utilization in construction materials was limited by fine particle size and high crystallization. This paper assesses the feasibility of reusing QS as siliceous materials to prepare a novel eco-friendly brick via press moulding and autoclave treatment. Three types of calcium sources including CaO, Ca(OH)2, and CaCO3 were employed to evaluate the binding efficiency with QS by using X-ray diffraction (XRD), thermogravimetry and differential scanning calorimetry (TG-DSC) and scanning electron microscopy (SEM). The effect of curing temperature, Ca/Si ratio and exposing time on the reaction activity of QS were estimated, and their roles on the microstructure and compressive strength of QS-based bricks were evaluated. Results demonstrate that the activity of QS can be effectively activated by Ca(OH)2. The morphology of the curing phases is determined by the dosage of QS. Moderating curing temperature (160 °C to 200 °C) and exposing time (≤15 h) contribute to higher reaction activity of QS and favor the generation of plate-like tobermorite and C-S-H, which induced a denser structure of QS-based autoclaved bricks and further promoted the strength development. In addition, 24 h of exposing time induced the transformation from tobermorite to xonotlite, prejudicing the strength of QS-based autoclaved bricks. Considering the performance and energy consumption, the optimum manufacturing parameters of QS-based autoclaved bricks are curing at 200 °C for 15 h, with a Ca/Si ratio of 0.83. Under this treatment, the compressive strength of QS-based autoclaved brick is as high as 49.9 MPa with a QS dosage of 63.5%. In addition, an assessment of the mobility of heavy metals associated with QS-prepared bricks reveals that autoclaved treatment can solidify heavy metals and QS-based autoclaved bricks can be regarded as an environmentally friendly building product. This research provides a foundational framework for the efficient utilization of QS through hydrothermal activation.