The modified Langevin noise formalism [A. Drezet, Phys. Rev. A 95, 023831 (2017); O. D. Stefano, S. Savasta, and R. Girlanda, J. Mod. Opt. 48, 67 (2001)] has been proposed for the correct characterization of quantum electromagnetic fields in the presence of finite-size lossy dielectric objects in free space. The main modification to the original one [T. Gruner and D.-G. Welsch, Phys. Rev. A 53, 1818 (1996); H. T. Dung, L. Kn\"oll, and D.-G. Welsch, Phys. Rev. A 57, 3931 (1998)] (also known as the Green's function approach for a bulk inhomogeneous lossy dielectric medium) was to introduce another fluctuating source in reaction to the radiation loss. Consequently, the resulting electric field operator is now determined by (i) boundary-assisted and (ii) medium-assisted fields on an equal footing, which originate from radiation and medium losses, respectively. However, due to the lengthy mathematical manipulation and complicated concepts, the validity of the modified Langevin noise formalism has not been clearly confirmed yet. In this work, we propose and develop a numerical framework for the modified Langevin noise formalism by exploiting computational electromagnetic methods. Specifically, we present utilization of the finite-element method to numerically solve plane-wave-scattering and point-source-radiation problems whose solutions are boundary-assisted and medium-assisted fields, respectively. We numerically validated the modified Langevin noise model calculating the Purcell factor of a two-level atom inside or outside a lossy dielectric slab. The proposed numerical framework is particularly useful for analyzing the dynamics of multilevel atoms near plasmonic structures or metasurfaces in the open space.