Brain tumors, capable of yielding fatal outcomes, can now be identified through MRI images. However, their heterogeneous nature introduces challenges and time-consuming aspects to manual detection. This study aims to design the optimal architecture, leveraging Convolutional Neural Networks (CNNs), for the automatic identification of brain tumor types within medical images. CNN architectures frequently face challenges of overfitting during the training phase, mainly attributed to the dual complexities of limited labeled datasets and complex models within the medical domain. The depth and width hyperparameters in these architectures play a crucial role, in determining the extent of learning parameters engaged in the learning process. These parameters, encompassing filter weights, fundamentally shape the performance of the model. In this context, it is quite difficult to manually determine the optimum depth and width hyperparameters due to many combinations. With Bayesian optimization and Gaussian process, we identified models with optimum architecture from hyperparameter combinations. We performed the training process with two different datasets. With the test data of dataset 1, we reached 98.01% accuracy and 98% F1 score values. With the test data of dataset 2, which has more data, 99.62% accuracy and F1 score values were obtained. The models we have derived will prove valuable to clinicians for the purpose of brain tumor detection.