Opening the black box of connectionist nets: Some lessons from cognitive science

Abstract

Connectionist networks are not simply opaque black boxes with useful and efficient computational properties. Rather they have important internal structure that must be harnessed if their full potential as a novel computing technology is to be realised. First, the importance and usefulness of opening the black box is discussed and research is reviewed on how internal representations have been studied and used in the Cognitive Science literature. In the second section, a simple method for the geometrical analysis of decision space is presented. This shows connectionist networks as transparent boxes in which their computational properties are clear. The paper finishes with an example of how decision space diagrams can be useful for investigating under what circumstances it is best to adapt old weights for use in novel tasks.