Soft rock is a new material that could be used for the improvement of Mu Us Sandy Land, China. It can be utilized for wind prevention and sand fixation, both of which are of great importance to ecological restoration aims and cultivated land replenishment in desert areas. Four treatments with different compound ratios of soft rock and sand—0:1 (CK), 1:5 (P1), 1:2 (P2), and 1:1 (P3)—were studied. Fluorescence quantitative PCR (qPCR) and high-throughput sequencing were used to analyze the structure and diversity of the bacterial community in the compound soil and its relationship with physical and chemical parameters in the soil. The results showed that in comparison to CK treatment, soil organic carbon (SOC), total nitrogen (TN), and NH4+-N increased significantly in the P1 treatment; available phosphorus (AP), available potassium (AK), and NO3−-N increased significantly in the P3 treatment. The bacterial gene copy number with P3 treatment was highest, showing a significant increase of 182.05% compared with the CK treatment. The three bacterial groups with the highest relative abundance at the phylum level were Actinobacteria, Proteobacteria, and Chloroflexi, accounting for more than 70% of the total population. The bacterial α diversity showed the same trend, the diversity and abundance indices of the P1 and P3 treatments were the highest, and the β diversity showed that the community structure of the two groups in these treatments were similar. norank_f__Roseiflexaceae and Gaiella (belonging to Actinobacteria) significantly differed with differing compound ratios in each treatment. NO3−-N, NH4+-N and SOC were the main factors affecting bacterial community structure, and had a significant positive correlation with Gaiella abundance. These species are known to play an important role in stabilizing the soil structure of sandy land. Overall, 1:5 and 1:1 compound soil mixtures were beneficial towards the microbial community of sandy land, which plays an important role in biological sand fixation. This study provides an important theoretical basis for the supplementation of arable land area and the improvement of sandy land productivity, and has a good promotion prospect.