A cavity-based X-ray free-electron laser (CBXFEL) is a possible future direction in the development of fully coherent X-ray sources. CBXFELs consist of a low-emittance electron source, a magnet system with several undulators and chicanes, and an X-ray cavity. The X-ray cavity stores and circulates X-ray pulses for repeated FEL interactions with electron pulses until the FEL reaches saturation. CBXFEL cavities require low-loss wavefront-preserving optical components: near-100%-reflectivity X-ray diamond Bragg-reflecting crystals, outcoupling devices such as thin diamond membranes or X-ray gratings, and aberration-free focusing elements. In the framework of the collaborative CBXFEL research and development project of Argonne National Laboratory, SLAC National Accelerator Laboratory and SPring-8, we report here the design, manufacturing and characterization of X-ray optical components for the CBXFEL cavity, which include high-reflectivity diamond crystal mirrors, a diamond drumhead crystal with thin membranes, beryllium refractive lenses and channel-cut Si monochromators. All the designed optical components have been fully characterized at the Advanced Photon Source to demonstrate their suitability for the CBXFEL cavity application.