Deep learning has been shown to be able to recognize data patterns better than humans in specific circumstances or contexts. In parallel, quantum computing has demonstrated to be able to output complex wave functions with a few number of gate operations, which could generate distributions that are hard for a classical computer to produce. Here we propose a hybrid quantum-classical convolutional neural network (QCCNN), inspired by convolutional neural networks (CNNs) but adapted to quantum computing to enhance the feature mapping process. QCCNN is friendly to currently noisy intermediate-scale quantum computers, in terms of both number of qubits as well as circuit’s depths, while retaining important features of classical CNN, such as nonlinearity and scalability. We also present a framework to automatically compute the gradients of hybrid quantum-classical loss functions which could be directly applied to other hybrid quantum-classical algorithms. We demonstrate the potential of this architecture by applying it to a Tetris dataset, and show that QCCNN can accomplish classification tasks with learning accuracy surpassing that of classical CNN with the same structure.