The adoption of Nash equilibrium (NE) in real-world settings is often impractical due to its too restrictive assumptions. Game theory and artificial intelligence provide alternative (relaxed) solution concepts. When knowledge about opponents' utilities and types are not available, the appropriate solution concept for extensive-form games is the self- confirming equilibrium (SCE), which relaxes NE allowing agents to have wrong beliefs off the equilibrium path. In this paper, we provide the first computational and learning study of the situations in which a two-agent extensive-form game is played by heterogeneous populations of individuals that repeatedly match (e.g., eBay): we extend the SCE concept, we study the equilibrium computation problem, and we study how these equilibria affect learning dynamics. We show that SCEs are crucial for characterizing both stable states of learning dynamics and the dynamics themselves.