We predict a large anomalous Nernst effect in the inverse Heusler compensated ferrimagnets Ti$_2$Mn$X$ ($X$=Al,Ga,In) with vanishing net magnetic moments. Though the net magnetic moment is zero, the Weyl points in these systems lead to a large anomalous Nernst conductivity (ANC) due to the lack of a magnetic sublattice that inverses the sign of the Berry curvature. In comparison to the noncollinear antiferromagnets Mn$_3$Sn and Mn$_3$Ge, the high ANC stems almost entirely from the Weyl points in this class of compounds, and thus, it is topologically protected. This work shows for the first time a large ANC with zero net magnetic moments in collinear systems, which is helpful for comprehensive understanding of the thermoelectric effect in zero-moment magnetic materials and its further applications.