Penicillin fermentation dreg (PFD) is a solid waste discharged by pharmaceutical enterprises in the fermentation production process. Due to the residual antibiotic of PFD, the risk of antibiotic resistance bacteria (ARB) generation should be considered in the disposal process. High-throughput quantitative PCR (HT-qPCR) and 16S rRNA gene sequencing were performed to investigate the effect of PFD on the dynamics of antibiotic resistance genes (ARGs) and bacterial community during a lab-scale soil experiment. After the application of PFD, the bacterial number and diversity showed an obvious decrease in the initial days. The abundances of Streptomyces and Bacillus, which are the most widespread predicted source phyla of ARGs, increased remarkably from 4.42% to 2.59%–22.97% and 21.35%. The increase of ARGs was observed during the PFD application and the ARGs carried by PFD itself contributed to the initiation of soil ARGs. The results of redundancy analysis (RDA) show that the shift in bacterial community induced by variation of penicillin content is the primary driver shaping ARGs compositions.