• The removal mechanism of O-containing functional groups were studied. • Effect of O-containing groups on floatibility characteristics was studied. • The ESP, bond order, and BDE of lignite molecular structure were calculated by DFT. • The essential reason for decrease of water content during HTD was explained. Hydrothermal treatment dewatering (HTD) is a potential means for efficiently and cleanly utilizing lignite due to its comprehensive modification, including dehydration and deoxidation.These processes are as tightly linked to the removal of O-containing functional groups in lignite. However, the O removal mechanism during the HTD process is not clear. It is important to deeply study this problem for the efficient application of upgraded lignite. Herein, density functional theory (DFT) simulations combined with experiments have been carried out to investigate the removal mechanism of O-containing functional groups. In addition, the HTD influence on the flotation characteristics was studied. The results showed that the carbon content and the calorific value increase after the HTD process, which improve the lignite quality. The oxygen content decreased from 28.55 to 16.67%, and the O/C atomic ratio decreased from 0.33 to 0.16 at 310 °C. The FT-IR results showed that the decrease in oxygen content is due to the cleavage and decomposition of methoxy, alcohol hydroxyl, and carboxyl groups. These results are consistent with the DFT results of bond orders and bond dissociation energies (BDE). The BDE order in the HTD process is: C O in methoxy < C O in alcohol hydroxyl < C C attached to carboxyl and carbonyl < C O in phenoxy ether < C O in phenolic hydroxyl < C O in carboxyl and carbonyl. The most stable chemical bonds are C O in carboxyl and carbonyl where the BDE values are over 550 kJ/mol. The large absolute electrostatic potential values are found to be around the O-containing functional groups. These O-containing functional groups very easily form hydrogen bonds with water, which results in stronger hydrophilicity and high-water content. Therefore, the removal of O-containing functional groups changes the surface characteristics. Besides, the contact angle of the upgraded lignite increases while the wettability decreases. Thus, the flotation characteristics of the upgraded lignite are significantly improved.