The study of quantum geometry effects in materials has been one of the most important research directions in recent decades. The quantum geometry of a material is characterized by the quantum geometric tensor of the Bloch states. The imaginary part of the quantum geometry tensor gives rise to the Berry curvature while the real part gives rise to the quantum metric. While Berry curvature has been well studied in the past decades, the experimental investigation on the quantum metric effects is only at its infancy stage. In this work, we measure the nonlinear transport of bulk MnBi2Te4, which is a topological anti-ferromagnet. We found that the second order nonlinear responses are negligible as required by inversion symmetry, the third-order nonlinear responses are finite. The measured third-harmonic longitudinal (Vxx3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xx}}^{3\\omega }$$\\end{document}) and transverse (Vxy3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xy}}^{3\\omega }$$\\end{document}) voltages with frequency 3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\omega$$\\end{document}, driven by an a.c. current with frequency ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\omega$$\\end{document}, show an intimate connection with magnetic transitions of MnBi2Te4 flakes. Their magnitudes change abruptly as MnBi2Te4 flakes go through magnetic transitions from an antiferromagnetic state to a canted antiferromagnetic state and to a ferromagnetic state. In addition, the measured Vxx3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xx}}^{3\\omega }$$\\end{document} is an even function of the applied magnetic field B while Vxy3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xy}}^{3\\omega }$$\\end{document} is odd in B. Amazingly, the field dependence of the third-order responses as a function of the magnetic field suggests that Vxx3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xx}}^{3\\omega }$$\\end{document} is induced by the quantum metric quadrupole and Vxy3ω\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$${V}_{{xy}}^{3\\omega }$$\\end{document} is induced by the Berry curvature quadrupole. Therefore, the quadrupoles of both the real and the imaginary part of the quantum geometry tensor of bulk MnBi2Te4 are revealed through the third order nonlinear transport measurements. This work greatly advanced our understanding on the connections between the higher order moments of quantum geometry and nonlinear transport.