Passive acoustic monitoring has been an effective tool for bird sound analysis. However, bird sounds often include cicada noise, which is an obstacle for investigating bird sounds. For example, cicada noise can result in large deviations of acoustic index, which will lead to the mismonitoring of species richness trends. Therefore, there is a critical need to filter cicada noise for helping bird sound analysis. We develop a novel end-to-end deep learning model, named CicadaNet for filtering cicada chorus from recordings containing bird sound. CicadaNet utilizes a convolutional encoder-decoder network to encode and decode acoustic features and a conformer module for global and local sequence modeling. We build a clean bird sound dataset and collect a large amount of real cicada noise data for model evaluation. We compare CicadaNet with current state-of-the-art deep denoising models and traditional denoising algorithms. Experimental results show that CicadaNet achieves the best denoising performance (SegSNR is improved by 9.59 dB and SI-SNR is improved by 20.08 dB when the noisy SNR = 0 dB). Meanwhile, CicadaNet achieves good performance for the real-time denoising of cicada noise. Furthermore, CicadaNet achieves bird species-independent noise reduction. We evaluate the effectiveness of CicadaNet for bird diversity survey. CicadaNet achieves the best performance, which can effectively eliminate the deviation caused by cicada noise to the acoustic index. CicadaNet can be easily extended to the cancellation of other environmental noise, and we propose it for the acoustic denoising of other vocalizing animals.