Non-technical losses cause substantial commercial concerns to distribution network operators (DNOs). 80% of NTLs are related to electricity theft, which contains various high-techs and is increasingly difficult to detect. Advanced metering infrastructure (AMI) has enabled supervised machine learning (ML) to detect the NTLs, which significantly improved the detection rates. A further advance in ML type of methods requires sufficient labeled datasets, which is usually not available. To address this, this article proposes a self-supervised detection method that extracts long-term consumption patterns to detect fraud in low-voltage networks, known as NTL detection contrastive predictive coding (ND-CP). Smart meter data sequences are fed into a one-dimensional convolutional neural network (1D-CNN) first. The gated recursive unit (GRU) data is then used to extract global information. After that, the output of the prediction from the GRU model is used to construct positive and negative sample pairs for contrastive learning. Eventually, a single-layer neural network classifier for detection is trained using the long-term features extracted by ND-CP. Experiments are conducted with real electricity consumption data to verify the effectiveness of the proposed method.