Hydrological studies are useful in designing, planning, and managing water resources, infrastructure, and ecosystems. Probability distribution models are applied in extreme flood analysis, drought investigations, reservoir volumes studies, and time-series modelling, among other various hydrological studies. However, the selection of the most suitable probability distribution and associated parameter estimation procedure, as a fundamental step in flood frequency analysis, has remained the most difficult task for many researchers and water practitioners. This paper explains the current approaches that are used to identify the probability distribution functions that are best suited for the estimation of maximum, minimum, and mean streamflows. Then, it compares the performance of six probability distributions, and illustrates four fitting tests, evaluation procedures, and selection procedures through using a river basin as a case study. An assemblage of the latest computer statistical packages in an integrated development environment for the R programming language was applied. Maximum likelihood estimation (MLE), goodness-of-fit (GoF) tests-based analysis, and information criteria-based selection procedures were used to identify the most suitable distribution models. The results showed that the gamma (Pearson type 3) and lognormal distribution models were the best-fit functions for maximum streamflows, since they had the lowest Akaike Information Criterion values of 1083 and 1081, and Bayesian Information Criterion (BIC) values corresponding to 1087 and 1086, respectively. The Weibull, GEV, and Gumbel functions were the best-fit functions for the annual minimum flows of the Tana River, while the lognormal and GEV distribution functions the best-fit functions for the annual mean flows of the Tana River. The choices of the selected distribution functions may be used for forecasting hydrologic events and detecting the inherent stochastic characteristics of the hydrologic variables for predictions in the Tana River Basin. This paper also provides a significant contribution to the current understanding of predicting extreme hydrological events for various purposes. It indicates a direction for hydro-meteorological scientists within the current debate surrounding whether to use historical data and trend estimation techniques for predicting future events with issues of non-stationarity and underlying stochastic processes.