Abstract One prominent feature of cognitive radio (CR) involves spectrum sensing (SS), which allows licensed primary users to remain unaffected by secondary users' ability to discover and exploit unoccupied frequency bands. Spectrum sensing enhances the use of spectrum in CR devices, increasing their adaptability and efficiency in wireless communication systems. The rise of wireless equipment and the advent of IoT technologies compound this need for flexibility. Over time, the fixed allocation of frequencies has led to inefficiencies and underutilization as bandwidth needs increase. Deep learning and artificial intelligence have improved spectrum sensing by increasing detection probability of primary users presence under noisy environments, enabling cognitive radio systems to respond intelligently to fluctuations in RF environments. This article is concerned with deep learning techniques for spectrum sensing and modulation categorization with CBRT structure, which combines convolutional neural networks (CNNs), bidirectional recurrent neural networks (BRNNs), and transformer networks (TNs) to improve spectrum sensing. CNNs are responsible for performing spectrum feature extraction; BRNNs are used to capture temporal dependencies; and TNs are good at long range dependencies. Better performance for this model is aimed by integrating the three architectures described. In the proposed work, we considered six digital modulation schemes, for spectrum sensing. The sensing of spectrum in this model is performed using the RadioML2016.10B open-source dataset and performance metrics like the Jaccard Index (JI), Fowlkes’s Mallows Index, and F1 Score. Modulation classification has been performed using MIGOU-MOD open-source dataset. The proposed model exhibits good detection probability and sensing error, unlike other methods at lower SNR.