Breast cancer identification is the first and foremost step in the journey of proper diagnosis and treatment of this disease. Therefore, many medical examinations and applications are devised and used including different approaches for breast imaging. For medical image analysis, nonparametric algorithm has been used. This work focuses on improving breast cancer recognition in mammogram images using a nonparametric approach based on image pixel intensities (IBCNP). A nonparametric approach is a new expression to deal with uncertainty. In this paper, the nonparametric approach is used to solve the uncertainty in breast cancer detection; uncertainty means no one knows for sure how to detect breast cancer without imaging, which creates an uncertainty for the oncologists. This paper investigates the feasibility of applying the nonparametric approach in mammography (x-ray breast imaging) to detect and hence diagnose and treat breast cancer. Analysing breast images is a very complicated process and requires distinguishing normal and malignant pixel intensities. Maximizing the shared data between image pixel intensities and the desired areas (infected region) is defined as the identification problem, subject to a constraint on the total size of the desired area borders. As part of this research, a comparative study was conducted on the proposed algorithm, Breast Cancer Cells Infestation based on a set of partial differential equations (PDEs), Incoherent Motion in Breast Cancer (IMBC), Global Dynamics of a Breast Cancer Competition Model using five simple differential equations (SDFs), and Estimation of Intensity Using Nonparametric Method (EINP) techniques. These five techniques of medical image segmentation are applied to 156 different mammogram images to obtain the exact measurement of the efficiency of the identification process. This process uses multiple performance metrics such as the overall mammogram image quality (Q), the dice similarity coefficient (DSC), Peak signal-to-noise ratio (PSNR) metric, Haussdorff Distance (HD), and probability of tumor detection (PTD). This paper assumes lack of knowledge of the probability densities in relation to pixel intensities of an image within a given area. It proposes an estimation of non-parametric density that formulates the theoretical data optimization problem by applying curve evolution methods and deriving the correlated gradient flows. It uses level-set techniques to achieve the resulting evolution. The algorithm was applied to 156 sets of different mammogram images from twelve groups with both malignant and/or normal. The experimental findings showed that the proposed algorithm is 94.937% accurately effective in breast cancer detection.