The CA3 region is central to hippocampal function during learning and memory and has a unique connectivity. CA3 pyramidal neurons are the targets of huge, excitatory mossy fiber synapses from DG axons and have a high degree of excitatory recurrent connectivity. Thus, inhibition likely plays an outsized importance in constraining excitation and shaping CA3 ensembles during learning and memory. Here, we investigate the function of a never-before studied set of dendrite-targeting, GABAergic neurons defined by expression of the synaptic adhesion molecule, Kirrel3. We discovered that activating Kirrel3-expressing GABAergic neurons specifically impairs memory discrimination and inhibits CA3 pyramidal neurons in novel contexts. Kirrel3 is required for DG-to-GABA synapse formation and variants in Kirrel3 are strong risk factors for neurodevelopmental disorders. Thus, our work suggests that Kirrel3-GABA neurons are a critical source of feed-forward inhibition from DG to CA3 during the encoding and retrieval of contextual memories, a function which may be specifically disrupted in some brain disorders.