Attributed graph clustering is a fundamental task in complex network analysis. Many existing graph clustering methods utilize graph representation learning techniques to learn node representations, subsequently applying K-means for clustering. Despite the significant attention and promising outcomes of graph contrastive learning in graph representation learning, we find two essential problems that need to be solved. (1) How to achieve augmentation for contrast that preserves the cluster structure of a given graph? (2) How to design an effective contrastive learning mechanism that collaborates with clustering? Therefore, we propose an attributed graph clustering method under the contrastive mechanism with cluster-preserving augmentation, integrating node representation learning and clustering into a unified framework. Specifically, we construct a contrasting view based on a generated kNN graph and edge betweenness centrality to preserve the cluster structure in the original graph. Meanwhile, a multilevel contrast mechanism based on pseudo-label-guided negative sampling is proposed to maximize the agreement between node representations in multiple latent spaces. We jointly optimize a specific clustering objective during the contrastive process, leading to refined cluster distribution directly in training. Extensive experiments on several datasets demonstrate that our proposed model consistently outperforms existing state-of-the-art methods on clustering.