In this paper, a class of non-autonomous neural networks with time-varying delays is considered. By using a new differential inequality and M-matrix, we investigate the positive invariant set and global attracting set of the networks without the assumption on boundedness of time delays or system coefficients. On this basis, we obtain sufficient conditions on the uniformly boundedness, the existence of periodic attractor and give its existence range for periodic neural networks. Furthermore, we offer a weight learning algorithms to ensure input-to-state stability, and give the state estimate and attracting set for the system. Our results can extend and improve earlier ones. Some examples and simulations are given to demonstrate the effectiveness of the obtained results.