Accurate classification of magnetic resonance imaging (MRI) images of brain tumors is crucial for early diagnosis and effective treatment in clinical studies. In these studies, many models supported by artificial intelligence (AI) have been proposed as assistant systems for experts. In particular, state-of-the-art deep learning (DL) models that have proven themselves in different fields have been effectively used in the classification of brain MRI images. However, the low accuracy of multiple classification of these images still leads researchers to conduct different studies in this field. Especially there is a need to develop models that achieve high accuracy on original images, and it is believed that this need can be met not only by DL models but also by classical machine learning (ML) algorithms. However, it is critical to choose the hyperparameters correctly for the hybrid use of ML algorithms with DL models. This study proposes a powerful new hybrid method to perform multiple classifications of brain tumors with high accuracy. This method also uses a novel convolutional neural network (CNN) model for feature extraction, and ML algorithms are used for feature classification. In addition, nine state-of-the-art CNN models are used for CNN performance comparison. The Bayesian optimization algorithm is used to obtain the optimal hyperparameter values of ML algorithms. The results obtained from the experimental studies show that the proposed hybrid model achieved 97.15% mean classification accuracy and 97% recall, precision, and F1-score values. Other hybrid models, including DarkNet19-SVM, DarkNet53-SVM, DenseNet201-SVM, EfficientNetB0-SVM, InceptionV3-SVM, NasNetMobile-SVM, ResNet50-SVM, ResNet101-SVM, and Xception-SVM, achieved mean classification accuracies of 95.01%, 95.58%, 96.87%, 97.01%, 95.3%, 95.01%, 96.3%, 95.87%, and 96.23%, respectively. Additionally, the proposed hybrid model exhibited remarkable time efficiency, accomplishing the classification process in a mere 67 min. Conversely, the model that exhibited the lowest time efficiency was the InceptionV3, with a processing time of 370 min. In terms of computational complexity, the EfficientNetB0 model is the most efficient. Despite the higher computational complexity of the proposed CNN model compared to some other models, it achieves the second-best classification accuracy. These results show that the proposed method performs better than previous studies on the same dataset. Especially in the classification problem, the optimized ML algorithms were superior to CNN classifiers. Finally, except for one, the proposed CNN model achieved better classification accuracies than the state-of-the-art CNN models.