The simulation of semiconductor detectors is a key tool for developping and studying their behavior. In general, simulations of CZT detectors assume the crystal to be perfect, meaning that its properties are uniform. However, structural defects appearing in the crystal during growth modify these properties. Moreover, dynamic phenomena like polarization can appear. In particular, the electric field inside the detector can be disturbed by bulk charges, which creates uncertainties on measurement of incident photon energy and on interaction position estimated by sub-pixel positioning. One of the main issues of a simulation considering these non-uniformities is its complexity, especially if multiple or evolving electric field distributions have to be considered. Hence, we have developed a model accepting electric field modifications and allowing to observe quickly the detector's response modifications with the electric field. We leveraged GPU to address such computational burden. Indeed, we can afford to consider more complex simulations as the computation time is reduced. In this study, we introduced different types of spatial defects which may be found in real CZT crystals (point-like, planar, etc.) to observe quickly and easily their impact on the detector's measurement, on both spatial and spectral response.