In the current work, the mechanical and thermal characteristics of environmentally friendly lightweight composites with high specific strength (defined as the ratio of strength to density) and low thermal conductivity are examined. To increase the sustainability of composites, ordinary Portland cement (OPC) was completely replaced with alkali-activated alumino-silicate by-products, including ground granulated blast furnace slag (GGBFS) and silica fume (SF). The micro-sized hollow fly-ash cenospheres (FAC), together with alkali-activated binders (AAB), are used to produce lightweight composites under ambient conditions. The novel composites had 28-day compressive strength ranging from 29.7 to 82.3 MPa, and dry densities ranged from 953 to 1566 kg/m3. The thermal conductivity of the optimal mix was 0.223 W/(m⋅K), which is lower than that of cementitious materials reported in the literature at the same specific strength. Based on the results obtained from the microstructural analysis, the high compressive strength associated with low thermal conductivity of FAC-modified AAB is attributed to (i) the co-existence of C-A-S-H/N-A-S-H phases is beneficial for the formation of a high-strength matrix, (ii) the presence of a strong interface between matrix and FAC, and (iii) the void sizes introduced is controlled below 50,000 nm.