High-power mid-infrared fiber lasers, featuring superior beam quality and good power-scaling ability, have a few important applications in material processing, medical surgery, and molecule spectroscopy. The high-power pump light combiner, as one of the key elements for constructing a mid-infrared fiber laser, is crucial for the laser performance. While some advanced side-pump combiners based on fluoride fiber have been reported in recent literatures, the thermal stability of the fluoride fiber combiner, which is closely-related to its power-scaling capability, is a long-living challenge. In this work, we demonstrate a high-power mid-infrared side-pump combiner with improved thermal stability, realized using the point-by-point fusion-splicing technique between a silica fiber taper and a piece of Er-doped fluoride gain fiber. The developed combiner exhibits a high coupling efficiency of ∼90%, supporting highly-stable operation at an incident pump power of up to 60 W. Using this combiner, we constructed a continuous-wave mid-infrared fiber laser which can deliver stably 4 W output power at 2.8 µm without using active cooling system. At this lasing power, the maximum input pump power is limited to 20 W to prevent fiber end-facet degradation, which can be further improved with the use of endcaps. This remarkable thermal stability renders the combiner great application potentials in constructing compact, robust, high-power fiber lasers at mid-infrared wavelengths.