Attributed graph clustering is a fundamental task in graph learning field. Because of the high-dimensional node features and the complex non-Euclidean graph structure, it is challenging for attributed graph clustering methods to exploit graph information. Recent studies on graph contrastive learning (GCL) have achieved promising results. However, existing GCL-based methods neither consider a clustering-friendly node representation nor a clustering-oriented loss function, resulting in inferior performance. To this end, we propose NCAGC, a neighborhood contrastive representation learning method for attributed graph clustering task. Specifically, NCAGC constrains the representation learning of similar nodes by a neighborhood contrast module to ensure the compactness in the latent space, thus facilitating the clustering task. Meanwhile, a contrastive self-expression module is present for learning a discriminative self-expression coefficient matrix, which is crucial for the subsequent subspace clustering. Moreover, the two designed modules are trained and optimized jointly, which benefits the node representation learning and clustering to achieve mutual refinement. Extensive experimental results on four attributed graph datasets demonstrate the superiority of NCAGC compared with 16 state-of-the-art methods, which surpasses the sub-optimal method on Cora dataset by 2.1%, 4.3%, and 3.7% in terms of ACC, NMI, and ARI, respectively. Our code and dataset is available at https://github.com/wangtong627/NCAGC-NeuroCom.