Quality of service (QoS) has been mostly applied to represent non-functional properties of web services and differentiate those with the same functionality. How to accurately predict unknown service QoS has become a hot research issue. Although existing researches have been investigated on temporal-aware service QoS prediction, conventional approaches are restricted to a couple of limitations. (1) most of them cannot well mine the time-series relationships and the interaction invocation information among users and services. (2) even although some sophisticated approaches make use of recurrent neural networks for temporal service QoS prediction, they mainly focus on the learning of user-service temporal relationship and have paid less attention to more effectively represent implicit features, resulting in low accuracy on service QoS prediction. To deal with the challenges, we propose a novel deep learning based approach called DeepTSQP to perform the task of temporal-aware service QoS prediction by feature integration. In DeepTSQP, we first present an improved temporal feature representation of users and services by integrating binarization feature and similarity feature. Then, we propose a deep neural network with gated recurrent units (GRU), learning and mining temporal features among users and services. Finally, DeepTSQP model can be trained by parameter optimization and applied to predict unknown service QoS. Extensive experiments are conducted on a large-scale real-world temporal QoS dataset WS-Dream with 27,392,643 historical QoS invocation records. The results demonstrate that DeepTSQP significantly outperforms state-of-the-art approaches for temporal-aware service QoS prediction in terms of multiple evaluation metrics.