In this present study, we have analyzed different types of X-ray solar flares (C, M, and X classes) coming out from different classes of sunspot groups (SSGs). The data which we have taken under this study cover the duration of 24 yr from 1996 to 2019. During this, we observed a total of 15015 flares (8417 in SC-23 and 6598 in SC-24) emitted from a total of 33780 active regions (21746 in SC-23 and 12034 in SC-24) with sunspot only. We defined the flaring potential or flare-production potential as the ratio of the total number of flares produced from a particular type of SSG to the total number of the same-class SSGs observed on the solar surface. Here we studied yearly changes in the flaring potential of different McIntosh class groups of sunspots in different phases of SC-23 and 24. In addition, we investigated yearly variations in the potential of producing flares by different SSGs (A, B, C, D, E, F, and H) during different phases (ascending, maximum, descending, and minimum) of SC-23 and 24. These are our findings: (1) D, E, and F SSGs have the potential of producing flares ≥8 times greater than A, B, C and H SSGs; (2) The larger and more complex D, E, and F SSGs produced nearly 80% of flares in SC-23 and 24; (3) The A, B, C and H SSGs, which are smaller and simpler, produced only 20% of flares in SC-23 and 24; (4) The biggest and most complex SSGs of F-class have flaring potential 1.996 and 3.443 per SSG in SC-23 and 24, respectively. (5) The potential for producing flares in each SSG is higher in SC-24 than in SC-23, although SC-24 is a weaker cycle than SC-23. (6) The alterations in the number of flares (C+M+X) show different time profiles than the alterations in sunspot numbers during SC-23 and 24, with several peaks. (7) The SSGs of C, D, E, and H-class have the highest flaring potential in the descending phase of both SC-23 and 24. (8) F-class SSGs have the highest flaring potential in the descending phase of SC-23 but also in the maximum phase of SC-24.