Parity-odd magnetoelectric multipoles such as magnetic quadrupoles and toroidal dipoles contribute to various symmetry-dependent magnetic phenomena and formation of exotic ordered phases. However, the observation of domain structures emerging due to symmetry breaking caused by these multipoles is a severe challenge because of their antiferromagnetic nature without net magnetization. Here, we report the discovery of nonreciprocal linear dichroism for visible light (~4% at 1.8 eV) in a magnetic quadrupole ordered phase of antiferromagnetic Pb(TiO)Cu4(PO4)4, which enables the identification of magnetic quadrupole domains of opposite signs. Symmetry considerations indicate that nonreciprocal linear dichroism is induced by the optical magnetoelectric effect, i.e., the linear magnetoelectric effect for electromagnetic waves. Using the nonreciprocal linear dichroism, we successfully visualize spatial distributions of quadrupole domains and their isothermal electric-field switching by means of a transmission-type polarized light microscope. The present work exemplifies that the optical magnetoelectric effect efficiently visualizes magnetoelectric multipole domains responding to external perturbations.