In the medical field, image segmentation is a paramount and challenging task. The head and vertebral column make up the central nervous system (CNS), which control all the paramount functions. These include thinking, speaking, and gestures. The uncontrolled growth in the CNS can affect a person’s thinking of communication or movement. The tumor is known as the uncontrolled growth of cells in brain. The tumor can be recognized by MRI image. Brain tumor detection is mostly affected with inaccurate classification. This proposed work designed a novel classification and segmentation algorithm for the brain tumor detection. The proposed system uses the Adaptive fuzzy deep neural network with frog leap optimization to detect normality and abnormality of the image. Accurate classification is achieved with error minimization strategy through our proposed method. Then, the abnormal image is segmented using adaptive flying squirrel algorithm and the size of the tumor is detected, which is used to find out the severity of the tumor. The proposed work is implemented in the MATLAB simulation platform. The proposed work Accuracy, sensitivity, specificity, false positive rate and false negative rate are 99.6%, 99.9%, 99.8%, 0.0043 and 0.543, respectively. The detection accuracy is better in our proposed system than the existing teaching and learning based algorithm, social group algorithm and deep neural network.