ABSTRACT The rapid development in computer technology plays an essential role in the research works for performing fast and accurate identification of flower species through the processing of flower images with the support of mobile devices that seeks more attention in the research areas. It is highly significant for maintaining the sensitivity of ecological balance, and therefore, the image-processing approaches have provided improved outcomes in recent days. To rectify these existing problems and improve the accuracy attains the flower classification, the CNN variants of ensemble techniques can be used for the dynamic ensemble selection of the CNN networks. This paper develops an enhanced ensemble deep learning-based flower classification model to get efficient classification results with optimal models. The pre-processing is done through the Contrast Limited Adaptive Histogram Equalization (CLAHE) and filtering techniques. The pre-processed images are considered for the optimal pattern extraction phase, where optimal hybrid patterns are extracted from the Local Binary Pattern (LBP) and Local Vector Pattern (LVP). Here, the optimal hybrid pattern extraction phase contains the optimization in it using the enhanced heuristic algorithm named Improved Rat Swarm Optimizer (IRSO). The flower classification is performed with an Adaptive Dynamic Ensemble Transfer learning-based Convolutional Neural Network (ADET-CNN). Here, the optimal models are selected with the support of the same RSO algorithm. The experimental results show a better efficacy of the developed flower classification framework.