A large decrease in permeability is often observed during the filtration of nano-sized colloids, while fouling is widely regarded as the main explanation for this phenomenon. The osmotic pressure or concentration polarization (CP) of colloids can also contribute to the flux decline. However, the contribution of CP to flux loss cannot be determined by the traditional CP model. In this study, the effect of fouling and CP/osmotic pressure on flux was distinguished. The CP values of polyethylene glycol (PEG) and silica-colloids were determined by the osmotic pressures near the membrane surface and in the feed. The CP induced by colloids accounted for 43–95% of the flux loss in our experiments. Silica exhibited higher CP values (127–460), compared to 7–71 for PEG. This was attributed to the slower back diffusion caused by the larger colloids, as evidenced by the diffusion coefficients of 4.30 × 10−11 m2/s for silica (10 nm) and 1.45 × 10−10 m2/s for PEG (2.9 nm). Although the CP was mitigated by increasing the cross-flow velocity, CP values of 31 and 250 were observed for PEG and silica at high Reynolds number of 7317, respectively. The experimentally obtained CP values were also compared with those calculated by the film diffusion model.