Ternary Cu2SnSe3 exhibits an inherent low symmetry and an unfavorable Seebeck coefficient. By strategic alloying with the highly symmetrical ZnSe, the transformation of a monoclinic structure into a tetragonal one was realized, yielding a higher band degeneracy in the electronic band. Therefore, the highest power factor in Cu2SnSe3-ZnSe solid solution reached 7.39 μW cm−1 K−2 at 773 K. Accompanied by this, the energy barrier for slip initiation during the stacking formation decreased, enabling a higher possibility of stacking faults and twins and a low lattice thermal conductivity as 0.79 W m−1 K−1 at 773 K for (Cu2SnSe3)0.6(ZnSe)0.4. Based on ZnSe alloying, the weaker covalent AgSe bonds are introduced to reduce the phonon velocities and facilitate mass fluctuation by Ag-substitution for Cu, resulting in a very low lattice thermal conductivity as 0.49 W m−1 K−1 at 773 K for (Ag0.2Cu1.8SnSe3)0.6(ZnSe)0.4. Eventually, an excellent ZT of 0.98 was achieved at 773 K for (Ag0.2Cu1.8SnSe3)0.6(ZnSe)0.4 through a progressive optimization on thermoelectric properties. Our work underscored the pivotal role of enhancing structural symmetry, increasing stacking nucleation probability and weakening covalent bonds in realizing excellent thermoelectric performance.