This work proposed a three-phase modelling framework using the convolutional neural network (CNN) method to predict the performance a ball mill based on the externally measurable variables in the milling process. The data of the model were generated from the discrete element method under different conditions, including acoustic emission (AE) signals, power draw and grinding rate. In the pre-training and training phases, the CNN model was able to predict particle size distributions and grinding rates with R2 higher than 0.92. The model was then applied to the large mill system and the results showed that the model maintained its performance in the new system with limited training datasets. The transfer learning of the model was tested by comparing the model with an untrained model and the results showed the loss error (MSE) of transfer model converged to a lower level within 20 epochs while the untrained model could only converge to a larger error after 400 epochs, indicating with the pre-trained model required far less training time and data for better prediction. The potentials and limitations of the model were also discussed.