Theoretical note: the C/T ratio in artificial neural networks

Abstract

This paper describes computer simulations of the effect of the C/T ratio on acquisition rate in artificial neural networks. The networks consisted of neural processing elements that functioned according to a neurocomputational model whose learning rule is consistent with information on dopaminergic mechanisms of reinforcement. In Simulation 1, three comparisons were made: constant C and variable T, variable C and constant T, and a constant C/T with variable C and T. In the last two comparisons, C was manipulated by changing the probability of reinforcement within the intertrial interval (ITI), in the absence of the conditioned stimulus (CS). Acquisition rate tended to increase with C/T, and the invariant ratio had no effect. In Simulation 2, C was manipulated by changing the ITI, with continuous reinforcement in the presence of the CS and no reinforcements in its absence. Results were comparable to those obtained in Simulation 1. Simulation 3 further explored the effect of the invariant ratio, but with larger absolute values of C and T, which slowed acquisition significantly. The results parallel some experimental findings and theoretical implications of the Gibbon-Balsam model, showing that they can emerge from the moment-to-moment dynamics of a neural-network model. In contrast to that model, however, Simulation 3 suggests that the effect of invariant C/T ratios may be bounded.