Abstract In this work, damages in composite materials are characterized using geometrical and Fourier-Hu shape descriptors. Digital images are acquired from the front and rear sides of composite materials after 5-, 6-, and 7-mm indentations. An anisotropic diffusion filter is used to remove the noise and to smooth the digital images. The global and local damages in the composite materials are segmented using K-Means (KM) clustering. Geometrical features and Fourier-Hu–based moments are calculated from the extracted regions. The global and local damages due to different indentations are classified using random forest (RF). Results show that the KM is able to segment the global and local damages with cluster centers three and four, respectively. The Fourier-Hu moment suggested that the damage dimension is high in the rear side compared with the front side. The Hu moment performed better compared with using geometrical features to characterize the damages. RF with both geometrical and Fourier-Hu moments performed better than the individual features. The image-based analysis performed in this study concludes effective characterization of damages; this framework can be applied in the nondestructive evaluation of composite materials.