Switching of the perpendicular exchange bias polarity using a magneto-electric (ME) effect of α-Cr2O3 was investigated. From the change in the exchange bias field with the electric field during the ME field cooling, i.e., the simultaneous application of both magnetic and electric fields during the cooling, we determined the threshold electric field to switch the perpendicular exchange bias polarity. It was found that the threshold electric field was inversely proportional to the magnetic field indicating that the EH product was constant. The high EH product was required to switch the exchange bias for the film possessing the high exchange anisotropy energy density, which suggests that the energy gain by the ME effect has to overcome the interfacial exchange coupling energy to reverse the interfacial antiferromagnetic spin.