Thermoelectric generation (TEG) using single-walled carbon nanotubes (SWCNTs), which generate electricity simply by floating on the surface of water, is promising and has novel features that are not found in conventional TEG systems. However, water-floating SWCNT-TEGs generate only a small voltage owing to the hydrophobic nature of SWCNTs. In this study, to increase their output voltage, we added α-cellulose, which is hydrophilic in nature, to the SWCNT films for enhancing water evaporative thermal cooling, leading to an increase in the temperature gradient in the TEGs. When the TEGs were exposed to artificial sunlight, the TEG with SWCNT/α-cellulose (0.6 g) films exhibited the highest output voltage of 1.0 mV, which was more than three times higher than that exhibited for the SWCNT-TEG without α-cellulose. This phenomenon can be explained by the hydrophilic nature of the films, which increases the amount of water pumped to the film surface and enhances the evaporative cooling effect as this water evaporates, thereby creating a larger temperature difference within the TEGs. The results show their significant potential as a power source for sensors such as those monitoring water flow in water environments.