The relationship between properties of recently very intensively studied artificial neural networks and real biological neural networks is discussed. It is indicated that there are many deep differences, so far. One is forced to model such systems on the basis of dynamical system theory. Besides it is clear that one has to think about stable steady states instead of equilibrium ones as appropriate memories. The necessary conditions of stability of equilibrium memory states are derived. This is promising to be of great practical meaning in applications of Hopfield-like nets of this type. On the other hand it seems that in biological neural nets memories would work on the basis of more complex attractors. Indications are made that most probably are strange attractors.