In this paper, a new experimental phenomenon is demonstrated. During the femtosecond laser filamentation, the generated terahertz (THz) pulse has been found to be strongly confined inside the plasma channel, reaching a spatial diameter of a few tens of micrometers. It has been attributed to the formation of a plasma negative dielectric waveguide induced by the transverse inhomogeneous plasma density distribution. The new experimental phenomenon will renew the understanding of the THz wave generation and propagation dynamics during the femtosecond laser and air plasma interaction. Due to this strong spatial confinement, THz electric field strength could be enhanced by orders of magnitude, potentially providing a new approach to performing THz nonlinear optics with low laser energy.