Soil formation and ecological rehabilitation is the most promising strategy to eliminate environmental risks of bauxite residue disposal areas. Its poor physical structure is nevertheless a major limitation to plant growth. Organic materials were demonstrated as effective ameliorants to improve the physical conditions of bauxite residue. In this study, three different organic materials including straw (5% W/W), humic acid (5% W/W), and humic acid-acrylamide polymer (0.2% and 0.4%, W/W) were selected to evaluate their effects on physical conditions of bauxite residue pretreated by phosphogypsum following a 120-day incubation experiment. The proportion of 2-1 mm macro-aggregates, mean weight diameter (MWD) and geometric mean diameter (GWD) increased following organic materials addition, which indicated that organic materials could enhance aggregate stability. Compared with straw, and humic acid, humic acid-acrylamide polymer application had improved effects on the formation of water-stable aggregates in the residues. Furthermore, organic materials increased the total porosity, total pore volume and average pore diameter, and reduced the micropore content according to nitrogen gas adsorption (NA) and mercury intrusion porosimetry (MIP) analysis, whilst enhancing water retention of the residues based on water characteristic curves. Compared with traditional organic wastes, humic acid-acrylamide polymer could be regarded as a candidate according to the comprehensive consideration of the additive amount and the effects on physical conditions of bauxite residue. These findings could provide a novel application to both Ca-contained acid solid waste and high-molecular polymers on ecological rehabilitation at disposal areas.