Purpose In this paper, community identification has been considered as the most critical task of social network analysis. The purpose of this paper is to organize the nodes of a given network graph into distinct clusters or known communities. These clusters will therefore form the different communities available within the social network graph. Design/methodology/approach To date, numerous methods have been developed to detect communities in social networks through graph clustering techniques. The k-means algorithm stands out as one of the most well-known graph clustering algorithms, celebrated for its straightforward implementation and rapid processing. However, it has a serious drawback because it is insensitive to initial conditions and always settles on local optima rather than finding the global optimum. More recently, clustering algorithms that use a reciprocal KNN (k-nearest neighbors) graph have been used for data clustering. It skillfully overcomes many major shortcomings of k-means algorithms, especially about the selection of the initial centers of clusters. However, it does face its own challenge: sensitivity to the choice of the neighborhood size parameter k, which is crucial for selecting the nearest neighbors during the clustering process. In this design, the Jaya optimization method is used to select the K parameter in the KNN method. Findings The experiment on real-world network data results show that the proposed approach significantly improves the accuracy of methods in community detection in social networks. On the other hand, it seems to offer some potential for discovering a more refined hierarchy in social networks and thus becomes a useful tool in the analysis of social networks. Originality/value This paper introduces an enhancement to the KNN graph-based clustering method by proposing a local average vector method for selecting the optimal neighborhood size parameter k. Furthermore, it presents an improved Jaya algorithm with KNN graph-based clustering for more effective community detection in social network graphs.