In this paper, we consider trajectory tracking control for autonomous surface vehicles (ASVs) with unknown boundary model uncertainties and external disturbances. The neural networks (NNs) and the sliding mode control (SMC) with a switched adaptive law are combined for the first time. The NNs are used to approximate model uncertainties and external disturbances of ASVs. The parameters of the robust SMC term first increase until the sliding variable reaches a quasi sliding mode which bound is associated with the parameter of the barrier function (BF). Then the BF is selected as the parameter of the robust term for the SMC strategy to estimate the NN approximation error and to constrain the sliding variable inside the predefined quasi sliding mode. One salient feature of our approach is that the robust control parameter is no more than approximation error of NNs in sliding and steady phases of SMC. Simulation studies are performed to illustrate the advantage of the proposed control method.