Emerging hybrid in-plane and out-of-plane devices have exposed the urgent demand for simultaneously in-plane and out-of-plane exchange biased layers in a single functional device. In this work, we have performed detailed studies on the manipulation of dual exchange bias in the IrMn/Co/Pt/CoPt/IrMn heterojunction. The in-plane exchange bias between IrMn and Co layers can be reversibly controlled by simultaneous application of colinear in-plane magnetic field and current pulse along x-direction. And the out-of-plane exchange bias between CoPt and IrMn layers can be manipulated by the application of out-of-plane external magnetic field along z-direction or colinear in-plane magnetic field and current pulse along x-direction. Moreover, thanks to the interlayer exchange coupling between Co/Pt/CoPt, a purely electrical induced spin-orbit torque switching can be used to manipulate the out-of-plane exchange bias. Based on this purely electrical manipulation of out-of-plane exchange bias, multistate nonvolatile memory has been demonstrated. Our work has revealed the promising application prospects of flexible manipulation of exchange bias in future multifunctional and multidimensional spintronic devices.