In order to explore the distribution characteristics of nonlinear mechanical behavior and the tendency of damage and failure of deep in situ coal-rock mass, a layered composite thin plate coal-rock mass model is established based on thin plate theory and finite element method. The continuity of interlayer stress and displacement is maintained by transfer matrix function, and the maximum shear stress is calculated by Mohr-Coulomb strength criterion. The distribution characteristics of stress, displacement, energy and maximum shear stress of coal-rock mass under static load are obtained. The analysis results are as follows : ① The torsional stress concentration areas of τxz and τyz are formed on the boundary of coal-rock interface, and the principal stress concentration areas of σx, σy and σz are formed in the center and center of the boundary, and the shear stress distribution of τxy is formed on the diagonal line. ② The maximum shear stress of the long axis has a strengthening effect on the shear failure, and each layer forms a stress state of alternating tension-shear-compression-shear. It first destroys in the middle of the coal-rock interface and boundary, and then destroys along the long side and short side. ③ The coal-rock mass presents a ′ X ′ -type shear zone along the radial and axial. The shear failure occurs first, followed by the principal stress compression failure. The cracks are preferentially developed in the direction perpendicular to the minimum principal stress, and the stepped ′ V ′ -shaped failure characteristics are formed under the action of shear and tensile stress.