Crowd counting plays an important role in public security. Estimating the number of people in an image with congested crowd accurately is a challenging task. The crowd counting method based on fully convolutional network can perform well in crowd image with complex scene. In this paper, to address the counting problems of occlusion,background clutter and perspective effect, we proposed a simple but effective method called Context-aware Multi scale Fusion Network(CMF Net).The CMF Net applied VGG network as backbone to extract coarse features. Then, three context-aware multi-scale fusion modules (CMFM) are adopted. Each CMFM consist of multi-scale feature extraction module (MEM) and context-aware feature extraction module (CEM). In addition, we propose adaptively dense connection to promoted information transmission in the counting network. Experiments on four datasets demonstrate that our network achieves competitive and effective results.