One of the frequently reported defects of RANS-based turbulence models is overestimation of turbulent kinetic energy production in high speed separated flow problems, which causes significant prediction errors. The correct estimation of such flow in thrust optimized parabolic nozzles extremely depends upon the accurate prediction of the onset of flow separation. In this paper, firstly, the significant error of conventional RANS-based turbulence models is shown to predict the onset of flow separation in this type of nozzles. Then, the prediction accuracy is improved through the modification of the essential parameters of the generalized k-ω (GEKO) turbulence model. It was found that modifying the separation and mixing parameters of the GEKO model to realize the turbulent kinetic energy production resulted in the accurate prediction of onset of flow separation at the extensive range of nozzle pressure ratios. Using this modified model with new coefficients reduced the error of about 30% of the k-ω-sst model in estimating the onset of flow separation. Also, the nozzle pressure value at which the transition from free shock separation (FSS) to restricted shock separation (RSS) occurs is well predicted by this approach. After strengthening the turbulence model, the flow physics has been investigated with increasing and decreasing nozzle chamber pressure. The length of the separation shock and reflected shock waves which impose the presence of FSS or RSS patterns and transitional phenomena are discussed. Our new findings show that unlike the transition from FSS to RSS, the inverse transition from RSS to FSS did not depend on the length of the separation and reflective shocks.