Natural soil and rock materials and the associated artificial materials have cracks, fractures, or contacts and possibly produce rock fragments or particles during geological, environmental, and stress conditions. Based on color gradient distribution, a digital image processing method was proposed to automatically recognize the outlines of fractures, fragments, and particles. Then, the fracture network, block size distribution, and particle size distribution were quantitatively characterized by calculating the fractal dimension and equivalent diameter distribution curve. The proposed approach includes the following steps: production of an image matrix; calculation of the gradient magnitude matrix; recognition of the outlines of fractures, fragments, or particles; and characterization of the distribution of fractures, fragments, or particles. Case studies show that the fractal dimensions of cracks in the dry mud layer, ceramic panel, and natural rock mass are 1.4332, 1.3642, and 1.5991, respectively. The equivalent diameters of fragments of red sandstone, granite, and marble produced in quasi-static compression failures are mainly distributed in the ranges of 20–40 mm, 25–65 mm, and 10–35 mm, respectively. The fractal dimension of contacts between mineral particles and the distribution of the equivalent diameters of particles in rock are 1.6381 and 0.8–3.6 mm, respectively. The proposed approach provides a computerized method to characterize quantitatively and automatically the structure characteristics of soil/rock or soil/rock-like materials. By this approach, the remote sensing for characterization can be achieved.