Pepper mild mottle virus (PMMoV) is a devastating viral pathogen of pepper (Capsicum annuum) but it is unclear whether and how peppers protect against PMMoV infection. The expression of the chloroplast outer membrane protein 24 (OMP24) of C. annuum was upregulated under PMMoV infection and it interacted with PMMoV coat protein (CP). Silencing of OMP24 in either C. annuum or Nicotiana benthamiana facilitated PMMoV infection, whereas overexpression of N. benthamiana OMP24 in transgenic plants inhibited PMMoV infection. Both C. annuum OMP24 (CaOMP24) and N. benthamiana OMP24 (NbOMP24) localized to the chloroplast and have a moderately hydrophobic transmembrane domain that is necessary for their localization. Overexpression of CaOMP24 induced stromules, perinuclear chloroplast clustering, and accumulation of reactive oxygen species (ROS), the typical defense responses of chloroplasts transferring the retrograde signaling to the nucleus to regulate resistance genes. The expression of PR1 and PR2 was also upregulated significantly in plants overexpressing OMP24. Self-interaction of OMP24 was demonstrated and was required for OMP24-mediated plant defense. Interaction with PMMoV CP interfered with the self-interaction of OMP24 and impaired OMP24-induced stromules, perinuclear chloroplast clustering and ROS accumulation. The results demonstrate the defense function of OMP24 in pepper during viral infection and suggest a possible mechanism by which PMMoV CP modulates the plant defense to facilitate viral infection.