The severe acute respiratory syndrome virus 2 (SARS-CoV-2) seriously impacted public health. The evolutionarily conserved viral chymotrypsin-like main protease (Mpro) is an important target for anti-SARS-CoV-2 drug development. Previous studies have shown that the eight N-terminal amino acids (N8) of SARS-CoV Mpro are essential for its dimerization, and are used to design inhibitors against SARS-CoV Mpro dimerization. Here, we established a simple readout assay using SDS-PAGE and Coomassie blue staining to measure inhibitory activity of N8 peptide derived from SARS-CoV-2 Mpro. To optimize its inhibitory effect, we then modified the side-chain length, charge, and hydrophilicity of the N8 peptide, and introduced a mutated Mpro recognition sequence. As a result, we obtained a series of potent peptide inhibitors against SARS-CoV-2 Mpro, with N8-A24 being the most efficient with an IC50 value of 1.44 mM. We observed that N8-A24 reduced Mpro dimerization with an IC50 value of 0.86 mM. Molecular docking revealed that N8-A24 formed hydrogen bond interactions with critical dimeric interface residues, thus inhibiting its dimerization and activity. In conclusion, our study not only discovers a series of peptide inhibitors targeting the SARS-CoV-2 Mpro dimerization, but also provides a promising strategy for the rational design of new inhibitors against COVID-19.