In the ore milling process, the ore particles are usually irregular polyhedrons, and the grinding media are often spherical. The need for accurate simulation of this process calls for a contact algorithm between the polyhedron and sphere, achieving a balance of accuracy, stability, and efficiency. For this purpose, we propose a polyhedron-sphere contact model based on the deepest point method. Experiments and simulation studies by discrete element method (DEM) are conducted to test this model's accuracy in a hexahedral box and a lab-scale horizontal drum with lifters. Then, systematic research is carried out for an industrial semi-automatic grinding (SAG) mill to explore the effect of particle shape. To reveal the advantage over the traditional spherical DEM model, the polyhedron-sphere (PH-SP) grinding system is compared with a pure spherical grinding system. Results are discussed from the motion behavior particles, power consumption, collision energy on ore and liner, and liner wear. The results show that the charging of particles in the PH-SP grinding system is blocked to a certain extent by polyhedral particles, both the power consumption and energy utilization efficiency increase somewhat, the large energy collision between ore and liner has increased significantly. The liner wear of the PH-SP grinding system is more severe than the pure spherical grinding system, which indicates that the new model is necessary for more accurate prediction of the grinding process.