The small rhizome Chinese herbal medicine harvesting machine is used for excavating the underground rhizomes of Chinese medicinal plants. The reliability of an existing machine of this type was found to be suboptimal due to high vibration levels, as confirmed by direct measurements. To remedy this issue, the differential equation of motion was derived, solved, and visualized using MATLAB software (R2017a). The impact of various parameters on the equation of motion was analyzed through both time and frequency domain plots, as well as experimental analysis. The parameters studied included the rotational speed of the tractor’s power take-off (PTO) shaft, the machine’s overall mass and stiffness, the transmission ratio, and the excitation force generated by the machine’s reciprocating parts. To reduce vibration in the non-resonant state and avoid resonance as the natural frequency changes, several modifications were necessary: the PTO speed needed to be controlled, the stiffness of the machine had to be increased, and the mass of the reciprocating parts had to be decreased. Additionally, expanding the transmission-ratio range of the operational machinery was essential. Sandbags were added to the machine’s frame to increase its overall mass. The above measures have reduced the vibration speed of the harvester during operation. The vibration speeds Vmax and VRMS of the harvester under both working and non-working conditions have decreased by half compared to their original values, reducing the occurrence of resonance in the harvester and effectively mitigating vibration damage, thereby enhancing operational reliability.