Two new non-centrosymmetric chalcogenides, ACuGa6S10 (A = Rb, Cs) have been successfully synthesized by an "iterative substitution" strategy based on chalcopyrite CuFeS2 structural template. Benefiting from the substitution of Fe3+ cations by Ga3+ cations, ACuGa6S10 (A = Rb, Cs) exhibit wide suitable band gap of 2.48 and 2.40 eV, respectively, which is about five times higher than their structure template CuFeS2, and the large second harmonic generation response (1.5 and 1.8 × AgGaS2). Combining theoretical calculation and structural analysis confirm that the [GaS4] tetrahedra make the main contribution on their good liner and nonlinear optical (NLO) performances. The "iterative substitution" strategy expands the design idea of materials and can lead to the discovery of a large number of IR NLO compounds.