The resemblance of crystallographic and magnetic structures of the quasi-two-dimensional iridates Ba2IrO4 and Sr2IrO4 to La2CuO4 points at an analogy to cuprate high-Tc superconductors, even if spin-orbit coupling is very strong in iridates. Here we examine this analogy for the motion of a charge (hole or electron) added to the antiferromagnetic ground state. We show that correlation effects render the hole and electron case in iridates very different. An added electron forms a spin polaron, similar to the cuprates, but the situation of a removed electron is far more complex. Many-body 5d4 configurations form which can be singlet and triplet states of total angular momentum that strongly affect the hole motion. This not only has ramifications for the interpretation of (inverse–)photoemission experiments but also demonstrates that correlation physics renders electron- and hole-doped iridates fundamentally different.