There is a growing interest in the application of thermoplastics and thermoplastic composites for lightweight structures, thus also in friction stir welding (FSW) as a suitable joining technology. With the transformation of FSW from metals to polymers, several new tooling concepts were developed; mainly various heating setups to add external thermal energy to the process were designed. However, current temperature control approaches do not consider the cooling of the process at high rotational speeds and the asymmetry of the process in thermomechanical conditions. In this paper, we present a stationary shoulder with fluid temperature control, which is also capable of cooling the process. Additionally, asymmetric temperature control conditions can be examined by applying different flow directions. For investigation, butt welds of PMMA sheets were welded and assessed using the weld morphology and tensile strengths. Different fluid temperatures, feed rates, and both flow directions were applied and led to different tensile strengths. It could be shown that the optimal control temperature is different for the advancing side (AS) and the retreating side (RS). By using a transverse flow, a heat transport from the warmer AS to the colder RS could be established, which improved the strength. Moreover, the tensile strength could be increased by cooling the process at a feed rate of 100 mm/min. To improve the effectiveness of the temperature control, a new shoulder with independent temperature-controlled areas is proposed.