In this paper, a graph-based density clustering framework is proposed that detects the boundary points of clusters rather than cluster exemplars in high density regions. The framework introduces the connection density to measure the density relationship between points, which depends on both a density metric and a distance metric, and a weighted graph is constructed based on the connection density. By cutting off edges with low connection density, points located at the boundary region become isolated points from the rest of the weighted graph, and each connected subgraph forms an initial cluster. For the generated isolated points, a graph-based label propagation strategy is designed, the stable point in initial clusters with higher connection density with an isolated point preferentially propagate labels. Finally, a novel connection density based clustering algorithm is proposed, called CDBC, which can automatically identify clusters of arbitrary shapes. The experimental results show that the proposed method outperforms other advanced clustering algorithms on several synthetic and real-world datasets.