Abstract Coral sand particles exhibit a wide range of shapes, which can be divided into four shapes, e.g., blocky, dendritic and rodlike, flaky, and shell debris. The particle shape of these mixtures is defined by the sphericity, concavity, aspect ratio, flatness and overall regularity, which ranges from 0 to 1. The effect of particle shape on the strength, crushing characteristics, and critical state parameter is systematically investigated through a series of triaxial drainage shear tests under different confining pressures. And the relationship between critical state parameters and mechanical parameters is established. The test results demonstrate the existence of an evident strain-hardening phenomenon in the stress–strain curve of coral sand, accompanied by a strain-softening phenomenon when the bias stress reaches its peak value. The sample is initially subjected to shear shrinkage, followed by shear expansion. The volumetric deformation of the coral sand decreased with increasing peripheral pressure. The particles are transformed from rough irregular shapes to smooth spheres as evidenced by an increase in the shape parameter. The greater the degree of irregularity in the shape of the particles, the more pronounced the resulting change in size reduction. In addition, the critical state parameter was found to be influenced by the shape of the coral sand particles and the mode of particle accumulation. The overall shear resistance of coral sand particles was found to depend on particle rearrangement in addition to particle surface roughness and interparticle friction. It is proposed that the general regularity critical state parameter equation relates the particle shape of coral sand to its critical state mechanical properties, which is of great importance to the practical application and research of coral sand in engineering, and provides an effective means of predicting mechanical properties granular materials.