Since the world's population is rising continuously, more cultivable land is being utilized for their dwellings. As a result, the amount of food supply is decreasing day by day. In order to address the food shortage, a proper plan and technological breakthroughs is must. Tomato is a kind of vegetable which has the healthy ingredients and essential for our daily food list. The proposed system suggests an IoT based tomato cultivation and pest management system, with the help of deep learning methods. In the IoT implementation, camera module and moisture sensor are used to collect images of tomato plant, soil condition respectively. Based on the moisture content, the water pump will supply the water when it necessary. Besides, the real-time images of tomato leaf will be sent to the server to identify and classify natural enemies like various insect species. In the proposed system seven types of pests are identified with the help of ten deep learning models like InceptionV3, Xception, InceptionResNetV2, MobileNet, MobileNetV2, MobileNetV3Large, MobileNetV3Small, DenseNet121, DenseNet169, DenseNet201. This study has trained with leaves and insects separately to identify whether an image from a tomato plant is insectoid or not. 458 images of pests and 912 images of leaves are utilized in the proposed architecture. The accuracy of classifying insects or leaves using DenseNet201 is 100 %. The highest accuracy of 94 % is obtained to classify the different insects using the DenseNet201 model.