A series of epoxy-based azo polymers was synthesized from a precursor polymer that was obtained by step-polymerization between 1,4-cyclohexanedimethanol diglycidyl ether and aniline. The azo polymers were prepared by post-polymerization azo-coupling reactions of the precursor polymer with several diazonium salts to introduce azo chromophores bearing different electron-withdrawing groups. Photoinduced colloid deformation and surface modulation were investigated by using self-assembled colloidal spheres and spin-coated films of the azo polymers upon irradiation with Ar+ laser. Rates of both colloid deformation and surface modulation showed a close correlation with the electron-withdrawing groups on the p-positions of the azobenzene moieties. The azo polymer bearing the carboxyl group as the electron-withdrawing substituent showed much faster rates for both the colloid deformation and surface modulation compared with the others in the series. For comparable cases, the photoinduced deformation rates were obviously faster for the newly synthesized polymer compared with an epoxy-based azo polymer reported before.