We apply the method of QCD sum rules to study the $s q \bar s \bar q$ tetraquark states with the exotic quantum number $J^{PC} = 3^{-+}$, and extract mass of the lowest-lying state to be $2.33^{+0.19}_{-0.16}$ GeV. To construct the relevant tetraquark currents we need to explicitly add the covariant derivative operator. Our systematical analysis on their relevant interpolating currents indicates that: a) this state well decays into the $P$-wave $\rho\phi/\omega\phi$ channel but not into the $\rho f_2(1525)/\omega f_2(1525)/\phi f_2(1270)$ channels, and b) it well decays into the $K^*(892) \bar K_2^*(1430)$ channel but not into the $P$-wave $K^*(892) \bar K^*(892)$ channel.