This paper investigates the stability problem of delayed neural networks (DNNs) with time-varying delay. In the stability analysis of DNNs, the stability conditions are usually in form of second-order polynomial functions with respect to the time delay, which are nonlinear matrices inequalities and difficult to deal with. In order to obtain the stability criteria in terms of linear matrices inequalities (LMIs), some quadratic function negative-definiteness lemmas (QFNDLs) are proposed and widely employed. However, the research about QFNDL is still facing two challenges: large conservativeness and difficulty of optimisation. To overcome the challenges, this paper proposes a parameter-dependent quadratic function negative-definiteness lemma (PDQFNDL) to find the negative-definiteness condition. Compared with existing QFNDLs, the proposed lemma has two advantages: (i) Two moving points are introduced and the whole search interval is divided into two parts by an adjustable parameter, which contribute to the less conservativeness of stability criteria. (ii) The search interval of the adjustable parameter is smaller than the existing QFNDLs and the time of optimisation process is decreased. The proposed lemma is utilised to derive the less conservative stability criteria for DNNs based on a suitable Lyapunov–Krasovskii functional (LKF). Finally, the effectiveness and superiority of proposed stability criteria are demonstrated by several numerical examples.