The carbon fibers were ultrasonically treated under nickel chloride solution, and nickel ions were attached to the surface of the carbon fibers, which replaces the nitric acid oxidation method used in the original electroless nickel plating. Electroless nickel plating was performed on the surface of the carbon fibers using a palladium-free technique successfully. Nickel-plated carbon fibers with a higher weight gain ratio than the original method was obtained. XPS was used to study the functional group changes on the surface of carbon fibers and the chemical valence of nickel element. The mechanism of ultrasonic treatment on the surface of carbon fibers was analyzed, and the effect of ultrasonic treatment on the weight gain rate of nickel-plated carbon fibers was discussed. The results show that the ultrasonic treatment under nickel chloride solution can significantly increase the content of polar functional groups on the surface of carbon fibers and can promote the bonding of the carboxyl group on the surface of the carbon fibers with the hydrated nickel ion to form CFCOONi. The formation of CFCOONi enhances the bonding strength between the nano-nickel particles and the carbon fibers, thereby increasing the bonding strength between the plating layer and the substrate. The concentration of nickel chloride solution and ultrasonic power during ultrasonic treatment has a great influence on the weight gain rate of carbon fibers after electroless plating. When the concentration is greater than 10 g/L, the higher the concentration, the smaller the weight gain rate. When the treatment concentration is less than 10 g/L, the concentration has little effect on the weight gain rate. The optimum treatment concentration obtained in the experiment was 7 g/L. As the ultrasonic power increases, the weight gain rate increases first and then decreases, and the optimal ultrasonic power is 80 W. Ultrasonic time does not have a large effect on the weight gain rate, but the too low ultrasonic time will cause a slight decrease in the weight gain rate.