The estimation of space-varying geological parameters is often not computationally affordable for high-dimensional subsurface reservoir modeling systems. The adjoint method is generally regarded as an efficient approach for obtaining analytical gradient and, thus, proceeding with the gradient-based iteration algorithm; however, the infeasible memory requirement and computational demands strictly prohibit its generic implementation, especially for high-dimensional problems. The autoregressive neural network (aNN) model, as a nonlinear surrogate approximation, has gradually received increasing popularity due to significant reduction of computational cost, but one prominent limitation is that the generic application of aNN to large-scale reservoir models inevitably poses challenges in the training procedure, which remains unresolved. To address this issue, model-order reduction could be a promising strategy, which enables us to train the neural network in a very efficient manner. A very popular projection-based linear reduction method, i.e., propel orthogonal decomposition (POD), is adopted to achieve dimensionality reduction. This paper presents an architecture of a projection-based autoregressive neural network that efficiently derives an easy-to-use adjoint model by the use of an auto-differentiation module inside the popular deep learning frameworks. This hybrid neural network proxy, referred to as POD-aNN, is capable of speeding up derivation of reduced-order adjoint models. The performance of POD-aNN is validated through a synthetic 2D subsurface transport model. The use of POD-aNN significantly reduces the computation cost while the accuracy remains. In addition, our proposed POD-aNN can easily obtain multiple posterior realizations for uncertainty evaluation. The developed POD-aNN emulator is a data-driven approach for reduced-order modeling of nonlinear dynamic systems and, thus, should be a very efficient modeling tool to address many engineering applications related to intensive simulation-based optimization.