In recent years, much work has been done on processing of wireless spectrum data involving machine learning techniques in domain-related problems for cognitive radio networks, such as anomaly detection, modulation classification, technology classification and device fingerprinting. Most of the solutions are based on labeled data, created in a controlled manner and processed with supervised learning approaches. However, spectrum data measured in real-world environment is highly nondeterministic, making its labeling a laborious and expensive process, requiring domain expertise, thus being one of the main drawbacks of using supervised learning approaches in this domain. In this paper, we investigate the utilization of self-supervised learning (SSL) for exploring spectrum activities in a real-world unlabeled data. In particular, we assess the performance of SSL models, based on the reference DeepCluster architecture. We carefully consider the current state-of-the-art feature extractors, taking into account the performance and complexity trade-offs. Our findings demonstrate that SSL models achieve superior performance regarding the feature quality and clustering performance compared to baseline feature learning approaches. With SSL models we achieve significant reduction of the feature vectors size by two orders of magnitude, while improving the performance by a factor ranging from 2 to 2.5 across the evaluation metrics, supported by visual assessment. Furthermore, we showcase how adapting the reference SSL architecture to domain-specific data is followed by a substantial reduction in model complexity up to one order of magnitude, without compromising, and in some cases, even improving the clustering performance.