AbstractThis study proposes an innovative deep learning model for use in a multispectral vision system comprising a complementary metal‐oxide semiconductor image sensor and a spectrometer. To ensure accurate color recognition, the deep learning model includes an embedded adaptive automatic color temperature correction engine. By using this color temperature correction engine, the multispectral vision system can intelligently compensate for lighting and chromatic variations. To evaluate the performance of the system, we created a nine‐dimensional data set using the IT8.7/2 color target. We then used this data set to train the deep learning model. Our deep learning model outperformed other lightweight deep learning models in experiments, making it suitable for deployment on edge devices and embedded systems. We tested the ability of the multispectral vision system to classify adulterated coffee beans into their respective classes. The overall accuracy rate was more than 99.3%, indicating that out proposed multispectral vision system is effective in identifying color differences. Considering its capabilities in agricultural screening, we suggest incorporating our adaptive automatic multispectral vision system into agricultural machines for the realization of Agriculture 4.0.