The present study explores mechanical, thermal, acoustic, and fire performance evaluation of locally available rice straw as waste bio-mass in foamed gypsum hollow blocks. Two-core hollow gypsum blocks with various agro-waste compositions have been studied in detail and presented. Aluminum sulphate and potassium alum in 1:1 ratio are used as foaming agents in gypsum blocks. The density of two-core hollow blocks varies from 1070 to 730 kg/m3 for 0–15 wt% rice straw addition. The compressive strength of solid cubes and two-core hollow blocks lies in the range of 4.6-1.4 MPa and 2.2–1.1 MPa respectively. Whereas, the thermal conductivity lies between 0.15 and 0.07 W/m.K. exhibiting excellent thermal insulation. The sound transmission class (STC) of straw based foamed gypsum varies over the range of 23 to 20 dB for 20 mm thick with 0–15 wt% agro-waste added samples. The 5 wt% foamed rice straw gypsum sample has shown resistance to the surface spread of flame and the wallette unit of 600 mm × 600 mm has 70.5 kN of load carrying capacity at 800 °C. The foamed agro-waste-based hollow gypsum blocks may be used as a potential building material for non-load bearing wall application owing to its high thermal insulation, moderate acoustic and excellent fire retardant properties.