The Finite Particle Method (FPM) is a significant improvement to the traditional Smoothed Particle Hydrodynamics method (SPH), which can greatly improve the computational accuracy of the entire computational domain. However, unstable calculation results and long computational time are still major obstacles to the development of FPM. Based on matrix decomposition, the fundamental equations of the traditional Finite Particle Method are rewritten and a Generalized Finite Particle Method (GFPM) is derived by introducing Lagrange-type remainder. By deriving and rewriting the fundamental equations, the GFPM method can be theoretically proven to be always stable. Numerical examples show that the GFPM method can utilize a smaller computational scale to achieve the same computational accuracy as the FPM method, with a corresponding reduction in computational time. Finally, the GFPM method is applied to a one-dimensional stress wave propagation problem and a one-dimensional heat conduction problem, and the computational results are compared with those of the SPH method and the FPM method, which verify that the GFPM has higher computational accuracy and stability.