Underwater sound speed plays a vital role in maritime safety. Based on the acousto-optic self-interference effect, we proposed a new method to measure underwater sound speed utilizing Raman–Nath diffraction, generated by the acousto-optic effect between an optical frequency comb and pulsed chirp signal. When the pulsed chirp travels between the measurement and reference arm in the experimental setup that we constructed, the same signal resulting from acousto-optic self-interference is produced. The time gap between the two identical signals represents the time interval. Thus, we can determine the time-of-flight using cross-correlation. The optical path difference between the two arms is double the flight distance of ultrasonic waves and can easily be obtained using femtosecond laser interferometry. The time gap and the distance can be used to measure sound speed. The experimental results show that the chirp signal improves the signal-to-noise ratio and expands the applicable time-of-flight algorithm. The waveform pulse width after cross-correlation is 1.5 μs, compared with 40 μs before. The time-of-flight uncertainty can achieve 1.03 ns compared to 8.6 ns before. Uncertainty of sound velocity can achieve 0.026 m/s.