The use of industrial by-products is required to replace cement to reduce carbon emission, and it is necessary to secure the strength of cementless binder to satisfy the required mechanical performance. The purpose of this study is to develop a new binder system composed of activated fly ash (FA) with CaO by adding magnesium nitrate ((Mg(NO3)2) as a new additive. To reveal the mechanism of strength enhancement, the influence of the dosage of Mg(NO3)2 on compressive strength, reaction products, and pore characteristics was observed. In addition, expanded perlite (EP) and expanded vermiculite (EV) were mixed in selected samples to develop a lightweight cementless composite. The results showed that adding Mg(NO3)2 significantly increased strength of CaO-activated FA system. In the reaction process, Mg(NO3)2 promoted the solubility of quicklime (CaO) to form more C-(A)-S-H and Ca–Mg–Al-(OH)-NO3-AFm phase and increased the initial concentration of silicon (Si) and aluminum (Al) from FA. The dense C-(A)-S-H formation enhanced mechanical strength by reducing porosity in binder matrix. When EP and EV were mixed in the selected binder, even if the strength decreased as the amount of EP and EV increased, the requirements for the strength and specific gravity of commercial autoclave aerated concrete (AAC) were satisfied.