Waste water pollution containing heavy metal ions has raised wide concern in environmental sustainability, calling for the development of high-efficiency and low-cost materials for waste water treating. Herein, a series of carbon fiber/Co-Al layered double hydroxide/zeolitic imidazolate framework-8 (CF/LDH/ZIF-8) composites were synthesized and used for the high-efficiency elimination of toxic chromium Cr (VI) from aqueous solutions. The morphology, phase composition, phase structure of the CF/LDH/ZIF-8 composites were investigated by Fourier transform infrared spectroscopy (FT-IR), X-ray diffraction (XRD), scanning electron microscopy (SEM), energy dispersive spectroscopy (EDS), transmission electron microscopy (TEM) and nitrogen adsorption-desorption methods, respectively. The LDH/ZIF-8 films on the CF showed a homogeneous micro/nano scale porous structure, which was composed of CoAl LDH nanoplates and ZIF-8 nanocrystals. The nanocomposites showed high specific surface area of 156.9 m2 g−1 and pore diameters distributed almost in a narrow range of 2.5–8.3 nm when the molar ratio of Al3+/Co2+ reached to 2:3. High-efficiency elimination of toxic chromium Cr (VI) by the CF/LDH/ZIF-8 composites was confirmed by the absorption test which showed the high absorption capacities of Cr (VI) (378 mg/g). With the presence of CF/LDH/ZIF-8 composites in solution, toxic chromium Cr (VI) can be removed quickly, and the removal efficiency approached 99.3% approximately. Further, the results of cyclic tests indicated that the hierarchical CF/LDH/ZIF-8 composites exhibit excellent cyclic stability and can be fast regenerated in NaOH at 25 °C, rendering a widely application in environmentally friendly and high efficiency material for waste water treatment.