Cooperation is the cornerstone of social stability and human development. In order to promote mutual cooperation among individuals, some researchers analyzed the important factors influencing individual behavior from the perspective of group selection, while others revealed the evolutionary mechanism of cooperative behavior in groups from the perspective of network reciprocity. However, group selection and network reciprocity actually work together and simultaneously drive individuals to cooperate with each other. Analyzing each mechanism in isolation provides an incomplete understanding of the interaction process. Inspired by this, we integrate the coupled effects of both group selection and network reciprocity on the behavior of individuals. We develop a structured public goods game model to study the evolution of individual cooperative behavior in multiple groups, where each individual can interact not only with intra-group individuals but also with inter-group individuals. Based on the fixed probabilities of multi-group selection, including intra-group and inter-group selection, we derive a general condition that promotes cooperation among individuals. Besides, we discuss the effects of the number of neighbors in a group, group size, and group size on the selection of cooperative behavior. Finally, we systematically compare our model with the well-mixed case, and the results show that a structured population enhances cooperation. Increasing the number of populations boosts the fixation probability of cooperation. To the best of our knowledge, this paper is the first to study the cooperative evolutionary dynamics of multi-group selection in structured populations through public goods games.