Eukaryotic Argonaute proteins (eAgos) utilize short nucleic acid guides to target complementary sequences for RNA silencing, while prokaryotic Agos (pAgos) provide immunity against invading plasmids or bacteriophages. The Sir2-domain associated short pAgo (SPARSA) immune system defends against invaders by depleting NAD+ and triggering cell death. However, the molecular mechanism underlying SPARSA activation remains unknown. Here, we present cryo-EM structures of inactive monomeric, active tetrameric and active NAD+-bound tetrameric SPARSA complexes, elucidating mechanisms underlying SPARSA assembly, guide RNA preference, target ssDNA-triggered SPARSA tetramerization, and tetrameric-dependent NADase activation. Short pAgos form heterodimers with Sir2-APAZ, favoring short guide RNA with a 5′-AU from ColE-like plasmids. RNA-guided recognition of the target ssDNA triggers SPARSA tetramerization via pAgo- and Sir2-mediated interactions. The resulting tetrameric Sir2 rearrangement aligns catalytic residue H186 for NAD+ hydrolysis. These insights advance our understanding of Sir2-domain associated pAgos immune systems and should facilitate the development of a short pAgo-associated biotechnological toolbox.