Human behavioral decision-making influences the gaming environment. In turn, changes in the gaming environmental impact individual strategic choices. However, there is scant exploration into how human behavioral decision-making coevolves with dilemma strength. Here, we propose a coevolutionary game model based on collective-risk social dilemma, where an increase in cooperators within the game group reduces the dilemma strength, and vice versa. Upon examining this coupled system, we find that the system is capable of achieving a relatively optimal state, wherein the population sustains a high level of cooperation and the dilemma strength remains at the lowest level. In addition, we have identified the conditions for the emergence of tristability and bistability in the coupled system and numerically validated our theoretical results. Furthermore, we find that the incorporation of institutional rewards not only promotes the appearance of the system's optimal state, where all individuals choose to cooperate and the dilemma strength is at its lowest level, but it also effectively averts the manifestation of the system's worst state, where all individuals resort to defection and the dilemma strength reaches its highest level. These findings illuminate how cooperation can be sustained when a dynamical coupling exists between individual decision-making and dilemma strength. Published by the American Physical Society 2024