Design of an intelligent bean cultivation approach using computer vision, IoT and spatio-temporal deep learning structures

Abstract

Crop health monitoring and weed removal are two crucial elements dictating efficient, productive and resilient cultivation. Due to frequent attacks by pest and pathogens, the crops become diseased resulting in degradation of the quality and quantity of the production. The process of continuous monitoring of crop health is challenging and requires the involvement of information and communication technologies (ICT). The outcome is precision agriculture where the Internet of Things (IoT) and Artificial Intelligence (AI) techniques are vital ingredients. The design of an integrated approach of precision agriculture based on IoT and AI is discussed here which is tailored for real time crop health monitoring and performs various other operations like weed detection, ambient air sensing, watering the vegetation automatically at regular intervals of time, spraying of pesticides etc. The proposed system is a combination of an IoT formed using sensors and devices, image processing and machine learning (ML)/ deep learning (DL) techniques confined to the cultivation of fifteen varieties of beans found in India. The work involves two intelligent learning models configured to capture spatio-temporal attributes of image samples and sensor inputs and for real time discrimination between healthy and diseased bean leaves, detection of weeds growing around the cultivation land and also for process control. The first approach employs a DL structure named EfficientNetB7 along with a Bidirectional Long Short Term Memory (BiLSTM) while the second method adopts a VGG16 with an integrated attention mechanism. Also experiments have been carried out using benchmark ML classifiers like Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbor (KNN), Multi-Layer Perceptron (MLP) and Time Delay Neural Network (TDNN) combined with feature extraction techniques. Segmentation methods have been used to separate out the diseased sections of the leaves which are then used as apriori labels for the classifiers to reinforce the previously known details of the bean varieties. Subsequently, the trained networks are tested with bean leaf samples collected from cultivation farms. Results show that our proposed DL models could accurately predict the health state of the bean leaves with less computation time. With an automated approach of bean leaf health discrimination, weed detection and process control, the cost effectiveness of the overall effort is enhanced. Further, the sensor pack also provides precise thresholds at which water sprinkling could be initiated resulting in water conservation.