A three-dimensional non-isothermal and crystal plasticity based phase field model was proposed to simulate the cyclic deformation of nano-polycrystalline NiTi shape memory alloys (SMAs) reflecting their one-way shape memory effect (OWSME) and stress-assisted two-way effect (SATWSME), and then the microscopic mechanism of the cyclic degradation of shape memory effect (SME) was systematically revealed. The proposed model was numerically implemented in a two-dimensional plane strain nano-polycrystalline NiTi SMA system, where each grain was divided into a grain-interior (GI) phase and grain-boundary (GB) phase, and the GI phase was set to be embedded in the GB phase. For the GI phase, a crystal plasticity based phase field model was newly established by considering four inelastic deformation mechanisms, i.e., martensitic transformation (MT), martensitic reorientation (MR), austenitic plasticity caused by dislocation slipping and martensitic plasticity caused by dislocation slipping and deformation twinning, simultaneously; for the GB phase, a visco-plastic model was used to describe its plastic deformation. The simulated results show that: during the cyclic deformation of nano-polycrystalline NiTi SMAs involving the OWSME or SATWSME, the local internal stress caused by the MR or MT can lead to the accumulation of plastic deformation, which further increases the fraction of the martensite variants with favorable orientations by inducing a specific internal stress field after cooling and then results in the accumulation of residual martensite phase after heating; the accumulation of the irrecoverable strain mainly from the martensitic twinning and the decrease of recoverable strain cause the cyclic degradation of OWSME, which is further dependent on the loading rate and loading level; the SATWSME of the SMAs can exhibit a cyclic strengthening during the cyclic loading with a relatively low constant stress, while a cyclic degradation with a relatively high stress, which results from the competition between the transformation strain and the irrecoverable strain mainly caused by the austenitic plasticity that increases gradually during the cyclic deformation involving the SATWSME. These new findings are very helpful to clarify the physical nature of the cyclic degradation of SME and then alleviate the functional fatigue of NiTi SMAs by modulating the material design.