Lymphoid restricted membrane protein (LRMP) is a specific regulator of the hyperpolarization-activated cyclic nucleotide-sensitive isoform 4 (HCN4) channel. LRMP prevents cAMP-dependent potentiation of HCN4, but the interaction domains, mechanisms of action, and basis for isoform-specificity remain unknown. Here, we identify the domains of LRMP essential for this regulation, show that LRMP acts by disrupting the intramolecular signal transduction between cyclic nucleotide binding and gating, and demonstrate that multiple unique regions in HCN4 are required for LRMP isoform-specificity. Using patch clamp electrophysiology and Förster resonance energy transfer (FRET), we identified the initial 227 residues of LRMP and the N-terminus of HCN4 as necessary for LRMP to associate with HCN4. We found that the HCN4 N-terminus and HCN4-specific residues in the C-linker are necessary for regulation of HCN4 by LRMP. Finally, we demonstrated that LRMP-regulation can be conferred to HCN2 by addition of the HCN4 N-terminus along with mutation of five residues in the S5 region and C-linker to the cognate HCN4 residues. Taken together, these results suggest that LRMP inhibits HCN4 through an isoform-specific interaction involving the N-terminals of both proteins that prevents the transduction of cAMP binding into a change in channel gating, most likely via an HCN4-specific orientation of the N-terminus, C-linker, and S4-S5 linker.