In this paper, we study the problem of passivity analysis of fractional-order neural networks (FONNs) with a time-varying delay. By using the Razumikhin fractional-order theorem, we first derive an improved sufficient criterion for asymptotic stability of FONNs with a bounded time-varying delay. Then, based on the proposed stability criterion and some auxiliary properties of fractional calculus, a delay-dependent condition is established to ensure the passivity of the considered system. These conditions are order-dependent and in the form of linear matrix inequalities, which therefore can be efficiently solved in polynomial time by using the existing convex algorithms. Some numerical examples are provided to show the effectiveness of the obtained results.