We study the fermionic T-duality symmetry of integrable Green-Schwarz sigma models on AdS backgrounds. We show that the sigma model on $AdS_5\times S^1$ background is self-dual under fermionic T-duality. We also construct new integrable sigma models on $AdS_2\times CP^n$. These backgrounds could be realized as supercosets of SU supergroups for arbitrary $n$, but could also be realized as supercosets of OSp supergroups for $n=1,3$. We find that the supercosets based on SU supergroups are self-dual under fermionic T-duality, while the supercosets based on OSp supergroups are not. However, the reasons of OSp supercosets being not self-dual under fermionic T-duality are different. For $OSp(6|2)$ case, corresponding to $AdS_2\times CP^3$ background, the failure is due to the singular fermionic quadratic terms, just like $AdS_4\times CP^3$ case. For $OSp(3|2)$ case, the failure is due to the shortage of right number of $\kappa$-symmetry to gauge away the fermionic degrees of freedom, even though the fermionic quadratic term is not singular any more. More general, for the supercosets of the OSp supergroups with superalgebra $B(n,m)$, including $AdS_2\times S^{2n}$ and $AdS_4\times S^{2n}$ backgrounds, the sigma models are not self-dual under fermionic T-duality as well, obstructed by the $\kappa$-symmetry.