Climate variability affects the changes in controlling diseases transferred by insects. An increase in the population, the growth of communities, and a lack of public health infrastructure bring about the return of diseases of which insects are carriers, one of the illness issues. Therefore, the disease control is significant to help reduce the burden on the government and strengthen the country's public health structure. This research proposes a novel approach to modeling dengue fever dynamics, we employ a fractional derivative model with the Atangana–Baleanu–Caputo derivative, which offers a more accurate representation of real-world disease dynamics compared to traditional integer-order models. Basic qualifications are proposed. Equilibrium points and basic reproduction numbers are analyzed. The next-generation matrix method is used to identify the transmission. Besides, parameter sensitivity analysis is performed to learn about factors affecting input parameter values' effects on the basic reproduction number. It was found that the most common parameter affecting the transmission was the biting rate of mosquitoes was 1. In addition, the existence and uniqueness of the solution are examined using the Banach fixed point theorem. The Toufik–Atangana method is used for the numerical examination of a fractional version of the proposed model. We compared different values of fractional-order α=0.965, 0.975, 0.985, 0.995 and 1 it was found that when the order of derivatives decreases, the transmission shall decrease accordingly. This research provides valuable insights for developing effective control strategies to reduce the burden of dengue fever and strengthen public health systems.