Complete documentation on the status of mammals is indispensable for appropriate conservation measures in protected areas. However, there is inadequate information on mammalian resources in the ecosystem of Gibe Sheleko National Park (GSNP). Thus, the study aimed to assess species diversity, abundance, and habitat association of medium‐ and large‐sized mammals in GSNP. We stratified the study area into five dominant habitat types, namely dense forest, wooded grassland, grassland, riverine forest, and farmland habitat types based on land cover and vegetation structures and further employed stratified random sampling technique across each habitat type. The sample transects covered 20% of the study area. Transect width ranged from 50 m to 400 m based on vegetation cover and visibility of mammals. The main data were collected via direct observation. Data were analyzed via chi‐square test and species diversity indexes. We recorded the total of 20 mammals species' those belong to 10 families of which 8 species were large‐sized and 12 species medium‐sized mammals. There were two IUCN vulnerable species, namely Hippopotamus amphibious and Panthera pardus, and two globally near‐threatened species, particularly Litocranius walleri and Caracal caracal in the study area. Dense forest held the highest species diversity of medium‐ and large‐sized mammals (H′ = 2.28) with the highest evenness index (J = 0.84). Riverine forest had the least diversity with uneven population distribution. Papio anubis was the most abundance species, whereas Caracal caracal was the least abundant in the study area. GSNP is home for threatened and spectacular mammals species'; hence, an appropriate conservation measure is mandatory to keep existing mammals species'.