Globally, tea production and its quality fundamentally depend on tea leaves, which are susceptible to invasion by pathogenic organisms. Precise and early-stage identification of plant foliage diseases is a key element in preventing and controlling the spreading of diseases that hinder yield and quality. Image processing techniques are a sophisticated tool that is rapidly gaining traction in the agricultural sector for the detection of a wide range of diseases with excellent accuracy. This study focuses on a pragmatic approach for automatically detecting selected tea foliage diseases based on convolutional neural network (CNN). A large dataset of 3330 images has been created by collecting samples from different regions of Sylhet division, the tea capital of Bangladesh. The proposed CNN model is developed based on tea leaves affected by red rust, brown blight, grey blight, and healthy leaves. Afterward, the model’s prediction was validated with laboratory tests that included microbial culture media and microscopic analysis. The accuracy of this model was found to be 96.65%. Chiefly, the proposed model was developed in the context of the Bangladesh tea industry.