In this work, a novel three-dimensional grain-based model based on the Discrete Element Method (GBM3D-DEM) was proposed, and then the granite’s internal structure was reproduced in a realistic way. The effects of the micro-strength and micro-elastic modulus of the transgranular contact on the macroscopic mechanical characteristics and micro-cracking behavior of the sample in the Brazilian splitting test were investigated. Moreover, the variation of the fracture energy required for a single intergranular/transgranular contact was discussed. The consistency between the experimental and numerical results indicates that the proposed GBM3D-DEM is reliable. Results indicate that as the micro-strength of transgranular contact increases, the slope of the load–displacement curves in the linear elastic stage increases. The external load value required the macro-scale fracture of sample increases. The energy required for the transgranular contact fracture increases. In the propagation process of the main crack, the grain boundary structure is more easily destroyed, increasing the intergranular crack number. When the micro-elastic modulus of the transgranular contact rises, the stress region in the disk sample decreases gradually. In this case, the maximum external load value that the sample can withstand decreases. The transgranular contact is more likely to be destroyed under loading. The numerical results based on the GBM3D-DEM can predict the energy required for macro-scale fracture of crystalline granite, which is of great significance to rock mechanics.