A novel approach is developed to implement the forward modeling of a potential field caused by prismatic grids on gridded observations with higher efficiency and less memory. Based on the spherical symmetry of gravitational potential, we summarize the parity, symmetry, and interconversion relations from derivatives and higher derivatives of gravitational potential, which reveal that there are vast repetitive calculation and storage redundancies in conventional forward modeling of a potential field for prismatic grids, especially for cubic grids. These properties help to reduce not only the size of a single kernel matrix but also the number of necessary kernels in multicomponent forward modeling of a potential field, such as a gradient tensor field or joint inversion of a potential field. Several experiments on the synthetic and the realistic Bishop models are presented to illustrate how these properties are used in the forward modeling of gravity and magnetic gradient tensors. Based on the proposed properties, a 2D fast Fourier transform is performed to accelerate the discrete deconvolution of the kernel and the model parameters. With the new method, one single kernel can be reduced to half the number of model parameters, and the memory requirement and time cost of our method can be reduced by 1/8 and 1/2, respectively, compared with previous works. In forward modeling of six gravity gradient tensor components, only two kernels need to be computed and stored, which are 1/3 of the conventional method. These examples indicate that, with high precision, our method requires less memory and less time than conventional methods. It shows potential in the fast large-scale inversion of gravity, magnetic, and gradient tensors with limited hardware resources.