The differential relaying scheme is universally adopted as the main protection for power transformer. In certain situations, like temporary over-fluxing, this differential protection may malfunction. This is due to unbalance in primary and secondary current, as the saturated core does not reflect equal ampere-turn on either side. This article presents detection of over-fluxing condition against the internal fault based on the fifth harmonic component of differential current. As per the estimated level of the fifth harmonic concerning the fundamental component of differential current, the pickup setting of percentage bias characteristic will be modified to avoid maloperation. Once the over-flux condition is sensed, the algorithm activates timer-based V/f protection to protect the equipment. The proposed algorithm is validated on PSCAD software and after the collection of data, MATLAB software is used for further validation. The proposed algorithm is validated under various test conditions like normal load, external fault, internal fault, and over-fluxing. It is observed that the algorithm accurately detects internal fault in the transformer or blocks the relay operation under momentary over-fluxing situations. Further, to assassinate the competency of the proposed scheme, a practical test is performed on a single-phase transformer available in laboratory. The data for various test cases are captured with the help of DSO, and the collected data has been migrated to computer where the proposed algorithm is examined. After analyzing all the results, it is cleared that the proposed algorithm can prevent transformer isolation during momentary over-fluxing conditions.