Apple Valsa canker (AVC) caused by the Ascomycete Valsa mali, seriously constrains the production and quality of apple fruits. The symptomless incubation characteristics of Valsa mali make it highly challenging to detect AVC at an early infection stage. After infecting the wound of apple bark, the pathogenic hyphae of AVC will expand and colonize the phloem tissue. Meanwhile, various enzymes and toxic substances released by hyphae cause the decomposition of cellulose and lignin, and the generation of poisonous secondary metabolites in bark tissue. However, these early symptoms of AVC are invisible from the bark’s appearance. Fortunately, Terahertz Spectral Imaging (ThzSI) technology with the advantage of penetrating, and fingerprinting is promising for detecting hidden or slight symptoms of the fungal infection. This study is a preliminary investigation of terahertz frequency-domain spectra for AVC in the early stage of infection. Healthy and two-week-infected apple tree branches were prepared for capturing ThzS images, and the spectral data were preprocessed by Multivariate scattering correction (MSC), Savitzky-Golay convolution smoothing (SG), and standard normal variate (SNV) respectively to remove data noise and improve data quality. Principal component analysis (PCA), competitive adaptive reweighted sampling (CARS), and random frog (RFROG) were employed to extract the spectral feature bands to eliminate redundant data and improve computational efficiency. Machine learning models were established based on the spectral features to detect AVC at an early infection stage, where 11 of them exhibited the best performance with F1-score of 99.72%. To further explore disease information in spatial spectra, imaging data were acquired using terahertz imaging technology. Based on imaging data, pseudo-color imaging, histogram equalization, and Otsu segmentation were employed to visualize early infection areas in apple barks. Furthermore, histogram feature (HF), shape feature (SF), and local binary pattern (LBP) extracted from terahertz spectral images were utilized to establish the SVM, RF, and KNN models. HF-SF-KNN and HF-SF-LBP-KNN with the best performance achieved F1-score of 98.82%. This study presents a preliminary application of terahertz spectral and imaging technology for early-stage AVC detection and demonstrates its feasibility. Additionally, it provides a new way to detect AVC, which expands the application of ThzSI technology in tree disease detection in orchards and lays the foundation for further research.