A deep transfer learning (deep-TL) classification model has been proposed to diagnose tomato leaf disease. The main challenge of inaccurate classification of a convolution neural network (CNN) model was the availability of the small-sized dataset. This model deals with the challenges like availability of small-sized and imbalanced datasets. The proposed Alex support vector machine (SVM) fused hybrid classification (ASFHC) model is based on fully fusion technology that avoids overfitting to classify the type of disease in tomato leaves. The proposed model achieves the best performance in terms of accuracy by data augmentation of the training data. It uses a pre-trained network for feature extraction with the modification of architecture by concatenating two layers FC6 and FC7 (fully connected layer), plus a linear SVM classifier for classification of the disease. The uniqueness of the research is although the dataset is not balanced, the performance of the model has achieved the maximum. Compared with VGG 16 and VGG 19, the proposed model (ASFHC) has been evaluated using different measuring parameters, indicating remarkable computation time for implementation in the internet of things (IoT) domain. The overall accuracy attained by the model is 99.62%.