A hybrid composite biopolymer of magnetic chitosan-fly ash/Fe3O4 (CS-FA/Fe3O4) was prepared to be an effective composite biosorbent for the removal of reactive orange 16 (RO16) dye from aqueous media. Various analytical techniques such as XRF, BET, XRD, FTIR, and SEM–EDX were utilized to characterize of CS-FA/Fe3O4 composite. The effects of adsorption process parameters namely adsorbent dose (A: 0.04–0.12 g), solution pH (B: 4–10), temperature (C: 30–50 °C), and time (E: 20–90 min) were optimized by using Box–Behnken design (BBD) in response surface methodology (RSM). The experimental results indicate that the highest RO16 removal was 73.1% by significant interaction between BC (p-value = 0.0002) and AD (p-value = 0.022). The optimum RO16 dye removal conditions were recorded at solution pH ~ 4, adsorbent dose (0.08 g), temperature (30 °C), and time (55 min). The adsorption process was well described by pseudo-second order (PSO) kinetic and Freundlich isotherm model. The adsorption capacity of CS-FA/Fe3O4 composite for RO16 dye was 66.9 mg/g at 30 °C. The mechanism of the RO16 dye adsorption included many interactions such as electrostatic, n–π interaction, H-bonding, and Yoshida H-bonding. Furthermore, the CS-FA/Fe3O4 composite exhibited a high ability to separate from the aqueous solution after adsorption process by external magnetic field.