We study the stability of the steady waves forced by a moving localised pressure disturbance in water of finite depth. The steady waves take the form of a downstream wavetrain for subcritical flow, but for supercritical flow there is a solution branch for each specified forcing term, which has two localised solitary-like solutions for each Froude number, a small-amplitude and a large-amplitude solution. Our main purpose is to numerically investigate the stability of these steady waves by simulating the unsteady development from appropriate initial conditions. We use a forced Korteweg-de Vries model valid in the transcritical regime for weak forcing, and a fully nonlinear boundary integral simulation. The simulations using the forced Korteweg-de Vries model are in good agreement with the fully nonlinear simulations when the Froude number is near unity for small pressure forcing, as expected. We find that the steady subcritical downstream wavetrain is stable. For supercritical flow, the small-amplitude solitary-like wave which has bifurcated from a uniform flow is stable, whereas the large-amplitude solitary-like wave which has bifurcated from a free solitary wave is unstable. Furthermore, here a difference between the forced Korteweg-de Vries model and the fully nonlinear simulations is revealed. For moderate and large pressure forcing, the forced Korteweg-de Vries model predicts a stable solitary wave moving away from the pressure forcing, while the nonlinear simulation shows that this wave evolves to a breaking wave.