A fruit monitoring system based on image processing technology and multi-layer neural network is proposed. The advantage of the proposed fruit monitoring system allows it to be remotely controlled by PCs and the graphical user interface (GUI) program by LabVIEW which has been designed for more intuitive and convenient operation of this system. In addition, the neural network can reduce nonlinearity of the system compared to the calculation based system. Therefore, experienced workers and novices can easily judge the ripeness of the fruits using the GUI program without necessarily going to the orchards. In this study, the color is used as a criterion to judge the maturity of tomatoes. Ripe tomatoes will appear to be red, while the unripe tomatoes will be green in color. The region of interest (ROI) function and Canny edge detection are applied to crop the image and remove the background, then the pixel data obtained are to supply the use of neural network. After that the maturity level of tomatoes is judged by the neural network. In laboratory test, 50 experiments have been down, 48 of which were successful, 2 of which failed, so the recognition rate was 96%. The experiments of this fruit monitoring system in the greenhouse on real growing tomatoes has been conducted. Therefore, 10 experiments on the red and green tomatoes has been conducted, respectively. As a result, the recognition rate of the red tomatoes is 100%, and recognition rate of the green tomatoes is 90%. The experimental results show that the proposed mobile fruit monitoring system has a very high recognition rate of accuracy.