We explored nonlinear effects within the context of tsunami waveform inversion, wherein Green's functions were linearly superimposed to estimate earthquake slips. We focused on these effects while developing a source model for the 2003 Tokachi–Oki earthquake off Hokkaido, Japan. A source model for this earthquake was developed based on linear tsunami waveform inversion using Green’s functions and tsunami waveforms observed at tide gauge stations. Subsequently, tsunami waveforms from the source were simulated at the stations using nonlinear long-wave theory and compared with those estimated by inversion. The comparisons demonstrated that the waveforms had a non-negligible discrepancy that was attributed to advection effects, even for the primary wave used in the inversion at the two stations. This result strongly suggests that advection effects should be considered in the source modeling of the 2003 earthquake based on tsunami waveforms observed by tide gauges. Based on these results, a new tsunami waveform inversion technique that incorporates linearly approximated advection effects and maintain the framework of linear tsunami waveform inversion using Green’s functions is proposed and applied. The proposed method successfully mimicked the advection effects during the 2003 tsunami, reproduced better tsunami waveforms, and developed a source model for the 2003 earthquake using these effects. The peak slip amount and seismic moment were greater in the source model with advection effects than those without the effects. This finding suggests that the values in the source models developed for other earthquake events without considering these effects may have been underestimated.Graphical abstract