Theory of neural information processing systems

Abstract

I INTRODUCTION TO NEURAL NETWORKS 1. General introduction 2. Layered networks 3. Recurrent networks with binary neurons II ADVANCED NEURAL NETWORKS 4. Competitive unsupervised learning processes 5. Bayesian techniques in supervised learning 6. Gaussian processes 7. Support vector machines for binary classification III INFORMATION THEORY AND NEURAL NETWORKS 8. Measuring information 9. Identification of entropy as an information measure 10. Building blocks of Shannon's information theory 11. Information theory and statistical inference 12. Applications to neural networks IV MACROSCOPIC ANALYSIS OF DYNAMICS 13. Network operation: macroscopic dynamics 14. Dynamics of online learning in binary perceptrons 15. Dynamics of online gradient descent learning V EQUILIBRIUM STATISTICAL MECHANICS OF NEURAL NETWORKS 16. Basics of equilibrium statistical mechanics 17. Network operation: equilibrium analysis 18. Gardner theory of task realizability APPENDICES A. Historical and bibliographical notes B. Probability theory in a nutshell C. Conditions for central limit theorem to apply D. Some simple summation identities E. Gaussian integrals and probability distributions F. Matrix identities G. The delta-distribution H. Inequalities based on convexity I. Metrics for parametrized probability distributions J. Saddle-point integration REFERENCES