In this paper, the propagation mechanism of an oblique straight crack has been studied theoretically, which reveals its mechanical characteristics under in-plane biaxial compression. Firstly, the stress components away from the boundary are derived based on the superposition principle. The normal stress components are strengthened and shear stress component is restrained compared to the uniaxial condition. Then the relationship between stresses and stress intensity factors is analyzed, and the effect of stresses on the strength of cracked rocks is discussed. The analysis of wing crack growth shows that the reliable experimental results are very demanding for sample preparation. Based on Mohr-Coulomb criterion and Mohr’s stress circles, the failure mechanism of cracked rocks is analyzed, and the physical meaning of some formulas is vividly displayed. Moreover, we study the relationship between friction angle θ0 and angle β, which determines the minimum compressive strength of cracked rocks. There are evidences that the increase of crack opening width leads to β0 (a value of β) away from the theoretical value determined by sliding crack model, so that the role of stress σx can no longer be ignored. Theoretically speaking, for an initially closed crack, we find that, for the first time, both wing crack growth and shear compression failure are more likely to occur when the angle β between 22.5 and 45 degrees combining the statistical results of Barton and Choubey (Rock Mech Rock Eng 10:1–54, 1977). As for an initially open crack, the characteristics of stress intensity factors and circumferential stresses are also discussed, especially when σ1 equals σ3. Finally, we study the effect of osmotic pressure on stresses and stress intensity factors, the weakening of the properties of crack surfaces by water is also considered, and the mechanical behavior of a rock sample with an oblique straight crack changes dramatically.