A high-efficiency spaceborne processor for hybrid neural networks

Abstract

Featuring with characteristics of convolutional neural network (CNN) and recurrent neural network (RNN), hybrid neural network (H-NN) has been widely applied within the field of remote sensing. In order to satisfy de mands of on-orbit processing that requires high throughput with restriction on power consumption, designing specific heterogeneous array processor therefore becomes one of the most effective ways fulfilling various tasks engaged in the above field. In this paper, a heterogeneous array architecture is proposed to support the hybrid neural network, based on the characteristics of various computation types in between different neural network module types and of dynamic computation burden among different layers. Firstly, a heterogeneous array structure consisting of different PE, PPE, RPE and LPE units is proposed, enabling strong flexibility and high throughput. Four types of operation units are used for operations of MAC, ReLU, pooling and nonlinear lookup-table. Secondly, a multi-level on-chip memory structure and access strategy supporting different access modes are proposed to reduce the bandwidth requirements of off-chip data access and to improve the computation efficiency. Thirdly, a management strategy of heterogeneous computing array is designed, which combines pipelining and parallel processing to support efficient mapping of different types of hybrid neural networks. The hybrid neural network processor based on 65 nm CMOS technique has a peak throughput of up to 1.96 TOPS. The implementation on models of AlexNet, LRCN, VGG19-LSTM and CLDNN can achieve the throughput of 1.92 TOPS, 1.89 TOPS, 1.93 TOPS and 1.84 TOPS, respectively. Compared with the similar neural network processor that is based on the same technology, the throughput of AlexNet model is increased by 76.4%. The peak power consumption of a single processor core is 824mW, to which the power consumption restriction of on-orbit AI platform is satisfied.