Nowadays, NOR-containing wastewater has placed huge pressure on global ecology. In this study, a chemically-modified chitosan-based polymer was cross-linked with magnetite to prepare a novel magnetic composite adsorbent named Fe3O4/CS-P(AM-SSS) for norfloxacin (NOR) removal. The preparation conditions were optimized by single factor experiments and response surface methodology. A series of characterization analyses were carried out on the morphology, structure, and properties of Fe3O4/CS-P(AM-SSS), verifying that Fe3O4/CS-P(AM-SSS) was successfully prepared. Batch adsorption experiments showed that NOR was efficiently removed by Fe3O4/CS-P(AM-SSS), with a broad pH applicability of 3–10, short adsorption equilibrium time of 60 min, maximum adsorption capacity of 268.79 mg/g, and high regeneration rate of 86% after eight adsorption-desorption cycles. Due to the three-dimensional network structure and abundant functional groups provided by modified chitosan polymer, the superior adsorption capability of Fe3O4/CS-P(AM-SSS) was achieved through electrostatic interaction, π-π stacking, hydrophobic interaction, and hydrogen bonding. Adsorption process was exothermic and well fitted by the pseudo-second-order kinetic model and the Langmuir isothermal model. The presence of cations had a slight inhibitory effect on NOR adsorption, while humic acid nearly had no effect. In model swine wastewater, 90.3% NOR was removed by Fe3O4/CS-P(AM-SSS). Therefore, with these superior characteristics, Fe3O4/CS-P(AM-SSS) was expected to be an ideal material for treating NOR-containing wastewater in the future.