A self-designed ultrasonic-assisted welding platform was built to improve the poor microstructure and properties of conventional TIG welded ferritic stainless steel. The ultrasonic vibration was transmitted to the weld pool through the base metal in the manner of point–surface contact in the optimal position after calculation. The results show that the coarse columnar grains in the welded joint can be transformed into very fine equiaxed grains under ultrasonic vibration, especially the coarse columnar grains near the fusion line where cliff-like refinement occurs. The maximum grain size in the weld seam is reduced from 420 μm to 260 μm, and the average size is reduced by 60%. At the same time, the grain orientation tends to be harmonized. The microhardness of the welded joint is greatly improved on the whole, and the softening of the heat-affected zone caused by grain coarsening is effectively inhibited. Compared with the welded joints without ultrasonic assistance, the tensile strength and yield strength can be increased by 61 MPa and 47 MPa, respectively, under 130 W ultrasonic vibration. By strengthening the weak part of the welded joint, the weldability and toughness reserve of 441 ferritic stainless steel can be significantly improved.