Information processing and Bayesian analysis

Abstract

Science involves learning from data. Herein this process of learning or information processing is considered within the context of optimal information processing, as in Zellner (1988,1991,1997). Information criterion functionals are formulated and optimized to provide optimal information processing rules, one of which is Bayes’ theorem. By varying the inputs and using alternative side conditions, various optimal information processing rules are derived and evaluated. Generally output information = input information for these rules and thus they are 100% efficient learning rules. When different weights or costs are associated with alternative inputs, “anchoring” like effects, much emphasized in the psychological literature are the results of optimal information processing procedures. Further, dynamic information processing results are reviewed and extensions noted. Last, some implications of the information processing approach for learning from data will be discussed.