Despite the rapid advance of unsupervised reconstruction models in online service fault diagnosis, existing methods still lead to frequent false positive or false negative alarms. Tracing to the cause, popular reconstruction frameworks may fall into an “identical shortcut”, where both normal and anomalous cases can be well-recovered and hence faults are escaped from monitoring. Moreover, regularly “concept drift” deployed by online services always leads to sequences distribution dramatically change, which is easily wrongly-detected as anomalies. In this work, we present SS-Attention to avoid two contradictory false alarms simultaneously in a unified framework. Specifically, we first revisit the frameworks of MLP, CNN, and transformer, and confirm the important role of self-attention in preventing the model from learning shortcut, which we therefore select as our backbone. Second, we come up with a sparse attention layer to help model capture more long-term dependencies and distinguish local “concept drift” from anomalies. Third, we propose a semi-attention layer that urges model only learn from normal regions and further avoid well-recovering the faults. We evaluate our model on three public online service datasets and surpass the state-of-the-art alternatives by a sufficiently large margin, achieving an average 95.91% F1-Score with an approx. 34.48% economy of training time.