Abstract The behavior of nanoparticle motion has a great influence on gas-liquid mass transfer. However, it has been very difficult to characterize the motion of nanoparticles from a micro view in mass transfer experiments. In this study, a novel method was proposed to investigate nanoparticle Brownian motion through the application of the total internal reflection fluorescence microscope in a self-designed sample (a quasi-static liquid micro-groove) and the mass transfer enhancement of nanoparticles. Nanoparticle movement behavior was photographed using an electron-multiplying charge coupled device, and 100 consecutive images were recorded using Micro-Manager software at a rate of 20 fps. The images were processed through the particle tracking velocimetry algorithm to calculate two-dimensional motion rates of nanoparticles caused by Brownian movement. It showed that nanoparticle loadings influenced the motion rates significantly, and the motion rates were larger with smaller particle sizes under the same operating condition. The mass transfer coefficients in the quasi-static gas-liquid mass transfer system were calculated and analyzed through microscopic measurement. Based on the above thought, three important non-dimensional numbers [Sherwood (Shp ), Reynolds (Rep ), and Schmidt (Scp ) numbers] for mass transfer theory were studied.