FormerLeaf: An efficient vision transformer for Cassava Leaf Disease detection

Abstract

Leaf diseases have become more prevalent in recent years due to climate change, increased growth of outdoor air pollutants, and global warming. They may severely damage crop yield, leading to detrimental effects on global food security. The timely and precise detection of leaf diseases is thus crucial for preventing their spread and ensuring the sustainability of agricultural production. In this paper, we introduce a transformer-based leaf disease detection model, namely FormerLeaf along with two optimization methods to enhance the model performance. In more detail, we propose the Least Important Attention Pruning (LeIAP) algorithm to select the most important attention heads of each layer in the Transformer model. It could reduce the model size up to 28% and accelerate the evaluation speed by 15% with about 3% accuracy enhancement. In addition, we employ the sparse matrix-matrix multiplication (SPMM) to calculate matrix correlation in the model. This reduces the model’s complexity from O(n2) to O(n2p), resulting in lowering training time by 10% while keeping a similar performance. The evaluation results on the Cassava Leaf Disease Dataset show that our proposal outperforms the state-of-the-art models in most cases.