Here, we report a systematic investigation in the sensitivity of tiny Ni-doping on magnetic properties of magnetoelectric CaBaCo4O7. Interestingly, just 0.5% Ni-doping results in noticeable change in the magnetic properties of CaBaCo4-xNixO7 (0 ≤ x ≤ 0.07). The ferrimagnetic state of CaBaCo4O7 is drastically suppressed above 1% Ni-doping. This result suggests high sensitivity of Ni-doping on magnetic properties of CaBaCo4O7. We have compared our observation with reported non-magnetic and magnetic doping in CaBaCo4O7 and provided a plausible explanation of exceptional sensitivity of Ni-doping. We observed a decrease in ferrimagnetic TC and coercive field HC along with a dramatic suppression of remanent magnetization Mr in CaBaCo4-xNixO7 (0 ≤ x ≤ 0.07). Ni-doping results in competing magnetic interaction between ferrimagnetic and antiferromagnetic phases. Unlike, the nonmagnetic doping replacement of Co2+ by magnetic Ni2+ in the ferromagnetic chain in kagome layer induces antiferromagnetic exchange with the adjacent cobalt spins following Goodenough-Kanamori rules. This results in reorientation of spin that modifies the ferrimagnetic ground state resulting in competing magnetic states. Ni substitution leads to disorder and disturbs the exchange interactions which could be responsible for competing magnetic state and broadening of magnetic transition in the doped samples.