Oil shale is a special energy source between coal and petroleum and is a potential alternative energy source. However, during oil shale mining, a large amount of Inorganic mineral impurities is produced. Thus, the grade of the oil shale is reduced, it can be effectively utilized by performing beneficiation through physical sorting to reduce impurities and upgrade the oil content. In this paper, the physical properties of oil shale were analyzed, beneficiation of 50-0 mm oil shale using a compound dry separator was proposed. The effect of vibration on the beneficiation process was emphatically investigated. The law of force between particles in different sorting areas and the effects of amplitude, frequency and angle of vibration on the energy law of different sorting area are systematically analyzed, The optimum operation parameters were obtained, and the sorting test was carried out. The results showed that when the force uniformity between the concentrate area and the tailings area is the best and the fluctuation range is minimum and the peak force of the particles corresponding to the concentrate area and the tailings area is 8.24 mN, 10.57 mN, 12.34 mN, 8.13 mN under the amplitude is 3.8–4.0 mm, frequency is 38-39hz. When frequency is 38 hz, Amplitude is 3.9 mm, the vibration acceleration of the concentrate area and tailings area are 24.78 m/s2, 17.89 m/s2, and the concentrate area was the main separation area, and the tailings area was mainly used to transport and discharge gangue. The separation tests were carried out under the optimum vibration parameters (A = 3.9 mm, f = 38hz, θ = 45°), The yields and oil content of the concentrate are 34.63% and 11.03%, while those of tailings are 65.37% and 1.01%, and the separating accuracy (probable error E) is 0.14 g/cm3, the actual separation density is 2.33 g/cm3. The vibration plays an important role in the compound dry separating oil shale efficiently.