Knowledge transfer is the basis for R&D teams and enterprises to improve innovation performance, win market competition and seek sustainable development. In order to explore the path to promote knowledge transfer within the R&D team, this study considers the bounded rationality and risk preference of individuals, incorporates prospect theory into evolutionary game, constructs a perceived benefits matrix distinct from the traditional benefits matrix, and simulates the evolutionary game process. The results show that, R&D personnel’s knowledge transfer decisions depend on the net income difference among strategies; only if perceived cost is less than the sum of perceived synergy benefit, perceived organization reward value, and perceived organization punishment value, can knowledge be fully shared and transferred within the R&D team. Moreover, R&D personnel’s knowledge transfer decisions are interfered by the irrational psychological factors, including overconfidence, reflection, loss avoidance, and obsession with small probability events. The findings help R&D teams achieve breakthroughs in improving the efficiency of knowledge transfer, thereby enhancing the capacity of enterprises for collaborative innovation.