The finite temperature phase diagram of QCD with two massless quark flavors is not yet understood because of the subtle effects of anomalous $U_A(1)$ symmetry. In this work we address this issue by studying the fate of the anomalous $U_A(1)$ symmetry in $2+1$ flavor QCD just above the chiral crossover transition temperature $T_c$, lowering the light quark mass towards the chiral limit along line of constant physical strange quark mass. We use the gauge configurations generated using the Highly Improved Staggered Quark (HISQ) discretization on lattice volumes $32^3\times8$ and $56^3\times 8$ to study the renormalized eigenvalue spectrum of QCD with valence overlap Dirac operator. We have implemented new numerical techniques that have allowed us to measure about $100$-$200$ eigenvalues of the gauge ensembles with light quark masses $\gtrsim 0.6$ MeV. From a detailed analysis of the dependence of the renormalized eigenvalue spectrum and $U_A(1)$ breaking observables on the light quark mass, our study suggests $U_A(1)$ is broken at $T\gtrsim T_c$ even when the chiral limit is approached.