Walnuts have rich nutritional value and are favored by the majority of consumers. As walnuts are shelled nuts, they are prone to suffer from defects such as mildew during storage. The fullness and mildew of the fruit impose effects on the quality of the walnuts. Therefore, it is of great economic significance to carry out a study on the rapid, non-destructive detection of walnut quality. Terahertz spectroscopy, with wavelengths between infrared and electromagnetic waves, has unique detection advantages. In this paper, the rapid and nondestructive detection of walnut mildew and fullness based on terahertz spectroscopy is carried out using the emerging terahertz transmission spectroscopy imaging technology. First, the normal walnuts and mildewed walnuts are identified and analyzed. At the same time, the image processing is carried out on the physical samples with different kernel sizes to calculate the fullness of the walnut kernels. The THz image of the walnuts is collected to extract the spectral information in different regions of interest. Four kinds of time domain signals in different regions of interest are extracted, and three qualitative discrimination models are established, including the support vector machine (SVM), random forest (RF), and k-nearest neighbor (KNN) algorithms. In addition, in order to realize the visual expression of walnut fullness, the terahertz images of the walnut are segmented with a binarization threshold, and the walnut fullness is calculated by the proportion of the shell and kernel pixels. In the frequency domain signal, the amplitude intensity from high to low is the mildew sample, walnut kernel, and walnut shell, and the distinction between walnut kernel, shell samples, and mildew samples is high. The overall identification accuracy of the aforementioned three models is 90.83%, 97.38%, and 97.87%, respectively. Among them, KNN has the best qualitative discrimination effect. In a single category, the recognition accuracy of the model for the walnut kernel, walnut shell, mildew sample, and reference group (background) reaches 94%, 100%, 97.43%, and 100%, respectively. The terahertz transmission images of the five categories of walnut samples with different kernel sizes are processed to visualize the detection of kernel fullness inside walnuts, and the errors are less than 5% compared to the actual fullness of walnuts. This study illustrates that terahertz spectroscopy detection can achieve the detection of walnut mildew, and terahertz imaging technology can realize the visual expression and fullness calculation of walnut kernels. Terahertz spectroscopy and imaging provides a non-destructive detection method for walnut quality, which can provide a reference for the quality detection of other dried nuts with shells, thus having significant practical value.