Diatomite has been verified as an economical adsorbent to treat wastewater. Unfortunately, limited by poor regeneration and an inert surface with less active silanol groups, the retrievable and efficient diatomite-based adsorbent needs to be further explored. Here, on the basis of the thorough activation treatments of the raw diatomite, we developed a diatomite-based adsorbent with a uniform assembly of Fe3O4 nanoparticles on the surface. The nanocomposite possesses a high specific surface area, the trait of magnetic separation, and more functional groups for adsorption. The maximum adsorption capacities of Pb2+, Ni2+, and Cd2+ at 314 K reached 0.97, 1.18, and 0.88 mmol/g, respectively, which were more competitive than those of previously reported diatomite-based adsorbents. The experimental data fitted well with the pseudo-second-order kinetic model and Freundlich isotherm model, demonstrating that the adsorption was mainly derived from the electrostatic interaction and chelation between the hydroxyl/carboxyl groups on multilayer surfaces and the hazardous metal ions. According to the analytical results of the adsorption isotherms and response surface optimization, the removal efficiency can exceed 94%, obtained by increasing the pH and reaction temperature. The activation-enhanced surface engineering can arouse the adsorption potential of a diatomite-based adsorbent and provide universal perspectives into the concept design of a nanocomposite adsorbent with lower cost recovery and higher adsorption efficiency.