Element-free Galerkin (EFG) method is applied for the first time to numerical modeling of additive manufacturing (AM) process, in which the superiorities of the EFG method in implementing adaptive computation and in constructing high order approximation are exploited. Second-order moving-least squares (MLS) approximation is constructed for both temperature and displacement. A non-linear coupled thermo-mechanical model considering moving heat sources, radiation boundaries, elastoplastic materials and temperature-dependent parameters is adopted. Adaptive coarsening based on the background integration mesh is developed to efficiently model the layered deposition process. In addition, an efficient integration scheme using background hexahedral elements is proposed to evaluate the domain integrals of the weak forms. Numerical results show that the developed method is able to effectively model the AM process with substantially reduced number of approximation nodes. The effects of laser power, scan speed and scan path on temperature and distortion are investigated. The improved accuracies due to the proposed integration scheme and the adopted second-order approximation are also demonstrated.