Introduction: The QRS complex is the most prominent waveform within the electrocardiograph (ECG) signal. The accurate detection of the QRS complex is an essential step in the ECG analysis algorithm, which can provide fundamental information for the monitoring and diagnosis of the cardiovascular diseases. Methods: Seven public ECG datasets were used in the experiments. A simple and effective QRS complex detection algorithm based on the deep neural network (DNN) was proposed. The DNN model was composed of two parts: a feature pyramid network (FPN) based backbone with dual input channels to generate the feature maps, and a location head to predict the probability of point belonging to the QRS complex. The depthwise convolution was applied to reduce the parameters of the DNN model. Furthermore, a novel training strategy was developed. The target of the DNN model was generated by using the points within 75 milliseconds and beyond 150 milliseconds from the closest annotated QRS complexes, and artificial simulated ECG segments with high heart rates were generated in the data augmentation. The number of parameters and floating point operations (FLOPs) of our model was 26976 and 9.90M, respectively. Results: The proposed method was evaluated through a cross-dataset test and compared with the sophisticated state-of-the-art methods. On the MITBIH NST, the proposed method demonstrated slightly better sensitivity (95.59% vs. 95.55%) and lower presicion (91.03% vs. 92.93%). On the CPSC 2019, the proposed method have similar sensitivity (95.15% vs.95.13%) and better precision (91.75% vs. 82.03%). Discussion: Experimental results show the proposed algorithm achieved a comparable performance with only a few parameters and FLOPs, which would be useful for the application of ECG analysis on the wearable device.