With the development of mission critical sensors and sensor networks (MC-SSNs), using MC sensor for monitoring the changes of soil moisture has become convenient and real time. However, the method used to process the data obtained by the MC sensor directly affects the monitoring effect. Although the past research methods have achieved certain results, they all need to extract data features. In this paper, we propose the soil moisture retrieval algorithm with time–frequency analysis and convolutional neural network (CNN) based on ultra-wideband radar echoes, which do not need to build feature database in advance. The algorithm transforms the soil echoes into time–frequency (TF) distribution patterns and utilizes the CNN algorithm to classify soil echoes with different soil volumetric water contents (VWCs). Wigner–Ville (WV) distribution and Choi–Williams (CW) distribution are the two methods used for time–frequency transform; VGGNet and AlexNet are, respectively, applied to classify the TF patterns with different VWCs. We totally construct four soil moisture retrieval systems (WV-AlexNet, CW-AlexNet, WV-VGGNet, and CW-VGGNet), and the echoes of 27 soil water contents with different signal-to-noise ratios (SNRs) are studied. The simulation results with raw data show that the correct recognition rate of soil VWCs can reach 100% when the soil echoes are at 10 dB (SNR). The WV-AlexNet system has the best recognition performance among the four systems.