This study aimed to investigate the influence of loosely bound water (LBW) on the compressibility of compacted fine‐grained soils and accurately determine the soil’s compression index. Four fine‐grained soils (i.e., heavy clay, heavy silt, lean clay, and lean silt) and a coarse‐grained soil were examined. The volumetric flask method was used to measure the LBW contents of the five soils. X‐ray diffraction (XRD) analysis was then performed to test the mineral compositions and help explain the reason why the LBW content varied between different soils. A concept of modified void ratio was proposed by assuming that LBW is a part of the solid phase in soil. Subsequently, consolidation tests and permeability tests were conducted on saturated compacted specimens. The results show that the compression indexes or permeability coefficients tend to be the same for the soils with identical initial modified void ratios. Consolidation tests were also carried out on the unsaturated compacted heavy silt of four different dry densities prepared at a water content higher than the optimum. They show that the compression of unsaturated soil occurs if pore air is discharged when the water content is less than the LBW content. This confirms the previous assumption that LBW can be regarded as a part of the soil solid phase. A modified compression index was deduced and implemented to predict the settlement of a road embankment. The result suggests that the modified compression index is capable of calculating the compression of fine‐grained soils, whose water contents are higher than their LBW contents.