On the assumption that the concrete is a multi-phases composite material composed of matrix, aggregate and the bonding interface, the failure progress of concrete is simulated. It requires that numerical method not only can simulate forces and stresses, the failure growth and contacts, but also can simulate the discontinuities such as joints and cracks,the large deformation after the failure and the propagation of multi-cracks. By adding the function of tracing the propagation of cracks,the Numerical Manifold Method proposed by Shi Genhua is developed which can simulate both the discontinuity of block system and the tensile or shear failure of intact block. The random aggregate model according to the Monte Carlo method and Fuller Graded Formula is carried out, and the concrete fracture process is simulated by the NMM. The strength and failure pattern are in good agreement with the experimental data, which shows that the method put forward and the program developed in this paper can effectively simulate the fracture process of concrete composed of multi-cracks.