When simulating fluid-structure interaction (FSI) problems involving moving objects, the implicit inverse distance weighting-immersed boundary method (IDW-IBM) developed by Du et al. [1] has to construct a large square correlation matrix and solve its inversion at each time step. In this work, a simplified inverse distance weighting-immersed boundary method (SIDW-IBM) is proposed to eliminate the intrinsic limitations in the original implicit IDW-IBM. Through error analysis using Taylor series expansion, a second order approximation can be derived, which allows us to approximate the large square correlation matrix into a diagonal matrix; thereby, we proposed the SIDW-IBM based on this second order approximation to explicitly evaluate the velocity corrections, where the needs to assemble the large correlation matrix and inverse it are circumvented. Owing to the fact that the inverse distance weighting interpolation removes the limitations in the Dirac delta function, the proposed SIDW-IBM has been successfully implemented on the non-uniform meshes to further improve the computational efficiency. The proposed SIDW-IBM is integrated with the reconstructed lattice Boltzmann flux solver (RLBFS) [2] to simulate some classic incompressible viscous flows, including flow past an in-line oscillating cylinder, flow past a heaving airfoil, and flow past a three-dimensional flexible plate. The good agreement between the present results and reference data demonstrates the capability and feasibility of the SIDW-IBM for simulating FSI problems with moving boundaries and large deformations.