This paper proposes a generalized fractional differential (GFD) mask which incorporates various fractional-order kernels such as power-law kernel, exponentional kernel, and Mittag-Leffler kernel, corresponding to Riemann-Liouville (RL)/Caputo, Caputo-Fabrizio (CF), and Atangana-Baleanu (AB) fractional-order differentials, respectively. Furthermore, a generalized fractional integral (GFI) mask incorporating RL and AB fractional integrals is put forward. The proposed generalized fractional masks provide a more flexible and appropriate tool for image enhancement and image denoising applications, and hence, avoids the construction of different masks for each kernel separately. Additionally, a generalized fractional integral and fractional differential based adaptive algorithm for image denoising (GFIFD-AA) is proposed. The proposedGFIFD-AA incorporates a novel noise detection method (NDM) for the detection of salt and pepper (SP) noise in images, which exhibits a profound advantage in avoiding misclassification of original pixels as noisy pixels when original image itself has some pixels with intensity values 0 or 255. The detected noisy pixels are processed by utilizing the proposed GFI-based adaptive mask (GFI-AM). The noise-free pixels are further updated by utilizing the proposed GFD-based adaptive mask (GFD-AM) so as to enhance the details of the image. Several standard and medical images of different characteristics are examined to evaluate the performance of the proposed approach on images affected by SP noise at various noise densities (i.e., 10%-95%). Furthermore, the performance of the proposed GFI mask is also tested for images affected by Gaussian noise and is compared against conventional fractional-order masks. The simulation results based upon several quantitative parameters validate the effectiveness of the proposed method against conventional methods.