Discrete element method (DEM) simulations of size-driven segregation of binary particles in a rotary drum were conducted to investigate the influence of filling level, size ratio, and rotation speed on the segregation performance. Segregation experiments with different filling levels were used to verify the DEM model and analyze the influence of filling levels on segregation. The granular bed was divided into five layers to study the axial segregation in the rotary drum. The total velocity fluctuation was used to discuss the granular behavior from a mesoscopic perspective. A segregation index was adopted to quantify the segregation performance with different parameters. It was found that binary particles in the drum with different filling levels have different segregation patterns. A core of small particles was formed in the middle of granular bed for the case with a higher filling level, while there is no core formed for the case with a lower filling level. Results obtained indicate that the size-driven segregation in the drum with lower filling level increases with the increase of size ratio. Within the rolling regime, the rotation speed has little influence on the final segregation index of particles.