Sensing is a basic ability of smart structures. Self-sensing involves the structural material sensing itself. No device incorporation is needed, thus resulting in cost reduction, durability enhancement, sensing volume increase and absence of mechanical property diminution. Carbon fiber renders electrical conductivity to a composite material. The effect of strain/damage on the electrical conductivity enables self-sensing. This review addresses self-sensing in structural composite materials that contain carbon fiber reinforcement. The composites include polymer-matrix composites with continuous carbon fiber reinforcement (relevant to aircraft and other lightweight structures) and cement–matrix composites with short carbon fiber reinforcement (relevant to the civil infrastructure). The sensing mechanisms differ for these two types of composite materials, due to the difference in structures, which affects the electrical and electromechanical behaviors. For the polymer–matrix composites with continuous carbon fiber reinforcement, the longitudinal resistivity in the fiber direction decreases upon uniaxial tension, due to the fiber residual compressive stress reduction, while the through-thickness resistivity increases, due to the fiber waviness reduction; upon flexure, the tension surface resistance increases, because of the reduction in the current penetration from the surface, while the compression surface resistance decreases. These strain effects are reversible. The through-thickness resistance, oblique resistance and interlaminar interfacial resistivity increase irreversibly upon fiber fracture, delamination or subtle irreversible change in the microstructure. For the cement–matrix composites with short carbon fiber reinforcement, the resistivity increases upon tension, due to the fiber–matrix interface weakening, and decreases upon compression; upon flexure, the tension surface resistance increases, while the compression surface resistance decreases. Strain and damage cause reversible and irreversible resistance changes, respectively. The incorporation of carbon nanofiber or nanotube to these composites adds to the costs, while the sensing performance is improved marginally, if any. The self-sensing involves resistance or capacitance measurement. Strain and damage cause reversible and irreversible capacitance changes, respectively. The fringing electric field that bows out of the coplanar electrodes serves as a probe, with the capacitance decreased when the fringing field encounters an imperfection. For the cement-based materials, a conductive admixture is not required for capacitance-based self-sensing.