Material anisotropy caused by crystal orientation is an essential factor affecting the mechanical and fracture properties of crystal materials. This paper proposes an improved ordinary state-based peridynamic (OSB-PD) model to study the effect of arbitrary crystal orientation on the granular fracture in cubic crystals. Based on the periodicity of the equivalent elastic modulus of a cubic crystal, a periodic function regarding the crystal orientation is introduced into peridynamic material parameters, and a complete derivation process and expressions of correction factors are given. In addition, the derived parameters do not require additional coordinate transformation, simplifying the simulation process. Through convergence analysis, a regulating strategy to obtain the converged and accurate results of crack propagation paths is proposed. The effects of crystal orientations on crack initiation and propagation paths of single-crystal materials with different notch shapes (square, equilateral triangle, semi-circle) and sizes were studied. The results show that variations in crystal orientation can change the bifurcation, the number, and the propagation path direction of cracks. Under biaxial tensile loading, single crystals with semi-circular notches have the slowest crack initiation time and average propagation speed in most cases and are more resistant to fracture. Finally, the effects of grain anisotropy on dynamic fractures in polycrystalline materials under different grain boundary coefficients were studied. The decrease in grain anisotropy degree can reduce the microcracks in intergranular fracture and the crack propagation speed in transgranular fracture, respectively.