Multi-plane crosstalk is a key issue affecting the quality of holographic three-dimensional (3D) displays. The time-multiplexing stochastic gradient descent (TM-SGD) method has been applied to solve the inter-plane crosstalk problem in multi-plane reconstruction. However, the inter-plane crosstalk increases greatly as the inter-plane interval decreases, and the optimization time increases greatly as the number of planes increases. In this paper, we propose a double-constraint stochastic gradient descent method to suppress inter-plane crosstalk in multi-plane reconstruction. In the proposed method, we use the mask to make the optimization process focus more on the signal region and improve the reconstruction quality. Meanwhile, we adopt a constraint strategy of phase regularization to reduce the phase randomness of the signal region and suppress inter-plane crosstalk. Numerical simulation and optical experiment results confirm that our method can effectively suppress the inter-plane crosstalk and improve the quality of the reconstructed planes at a lower inter-plane interval. Moreover, the optimization time of our method is almost 4 times faster than that of TM-SGD. The proposed method can contribute to the realization of tomographic 3D visualization in the biomedical field, which requires the reconstruction of multiple tomographic images without inter-plane crosstalk.