This paper presents a systematic procedure for determining fracture toughness of an anisotropic marble using the diametral compression test (Brazilian test) with a central crack on the discs. Additionally, a novel formulation to increase the accuracy in Stress Intensity Factor (SIF) calculations using Boundary Element Method (BEM) is applied to determine the stress intensity factors and the fracture toughness of anisotropic rocks under mixed-mode loading. The numerical results show that the SIFs for both isotropic and anisotropic problems are in good agreement with those reported by previous authors. The marble with clear white-black foliation from Hualien (in eastern Taiwan), was selected for the Brazilian tests. Diametral loading was conducted on the Cracked Straight Through Brazilian Disc (CSTBD) specimens to evaluate their fracture toughness. In addition, a new fracture criterion was developed to predict pure mode I, pure mode II or mixed mode (I–II) fracture toughness of the anisotropic marble. The new fracture criterion is based on the examination of mode I, mode II and mixed mode (I–II) fracture toughness for different crack angles and anisotropic orientation.