We consider a time reversal symmetry (TRS) broken Kane-Mele model superimposed with Haldane model and chart out the phase diagram using spin Chern number to investigate the fate of quantum anomalous Hall insulator (QAHI) and quantum spin Hall insulator (QSHI) phases. Interestingly, in addition to QSHI and QAHI phase, the phase diagram unveils quantum anomalous spin Hall insulator (QASHI) phase where only one spin sector is topological. We also find multicritical points where three / four topological phase boundaries coalesce. These topological phases are protected by an effective TRS and a composite anti-unitary particle-hole symmetry leading to remarkable properties of edge modes. We find spin-selective, spin-polarized and spin-neutral edge transport in QASHI, QSHI and QAHI phases respectively. Our study indicates that the robustness of the topological phase mainly depends on the spin gap which does not necessarily vanish at the Dirac points across a topological phase transition. We believe that our proposals can be tested in near future using recent experimental advancements in solid state and cold atomic systems.