Tree nuts are rich in nutrients, and global production and consumption have doubled during the last decade. However, nuts have a range of quality defects that must be detected and removed during post-harvest processing. Tree nuts can develop hidden internal discoloration, and current sorting methods are prone to subjectivity and human error. Therefore, non-destructive, real-time methods to evaluate internal nut quality are needed. This study explored the potential for VNIR (400–1000 nm) hyperspectral imaging to classify brown center disorder in macadamias. This study compared the accuracy of classifiers developed using images of kernels imaged in face-up and face-down orientations. Classification accuracy was excellent using face-up (>97.9%) and face-down (>94%) images using ensemble and linear discriminate models before and after wavelength selection. Combining images to form a pooled dataset also provided high accuracy (>90%) using artificial neural network and support vector machine models. Overall, HSI has great potential for commercial application in nut processing to detect internal brown centers using images of the outside kernel surface in the VNIR range. This technology will allow rapid and non-destructive evaluation of intact nut products that can then be marketed as a high-quality, defect-free product, compared with traditional methods that rely heavily on representative sub-sampling.