Iterative Pruning Research Articles

The Adaptive Network Architecture for a Time-constrained Environment

This paper proposes an adaptive network architecture called Hybrid SOLAR (Supervised One-shot Learning Algorithm for Real number inputs), which is a hybrid algorithm between a one-shot network construction algorithm and an iterative pruning algorithm. Hybrid SOLAR determines a network structure in two stages. In the first stage, Hybrid SOLAR requires only a single presentation of training examples to construct the network and learning is finished. In the second stage, the network prunes redundant weights to improve the generalization ability. Thus, Hybrid SOLAR retains the advantages of those two algorithms. It needs only a single presentation of the training set for learning and the generalization ability is satisfactory.

Iterative pruning in second-order recurrent neural networks

An iterative pruning method for second-order recurrent neural networks is presented. Each step consists in eliminating a unit and adjusting the remaining weights so that the network performance does not worsen over the training set. The pruning process involves solving a linear system of equations in the least-squares sense. The algorithm also provides a criterion for choosing the units to be removed, which works well in practice. Initial experimental results demonstrate the effectiveness of the proposed approach over high-order architectures.