The circumstances that favor or disfavor habitat specialization in species are not well understood, despite a vast literature on particular cases. The taxonomic, phylogenetic and geographical distribution of limpet-like gastropods specialized for life on the coiled shells of other gastropods and hermit crabs were surveyed. Predation-related traits were measured to evaluate the refugial status of shells as habitats from predation. There are 41 shallow-water marine shell-specialized limpets, all members of the Lottiidae, Calyptraeidae and Hipponicidae, which account for 3.5% of the global diversity of shallow-water marine limpet-like gastropods. These species are distributed mainly along tropical and some temperate coastlines but are notably few in the highly diverse Indo-West Pacific. The northeast Pacific stands out as the region with the most diverse and most host-specific shell-dwelling limpets. As expected, limpets on shells have shells with little or no external sculpture, all traits associated with low predation risk. Hermit crabs are hosts to specialized limpets living inside their shells, but no limpet specializes on the exteriors of hermit-crab shells. Immobile encrusting and shell-extending epibionts (hydrozoans, anthozoans, sponges and bryozoans) often specialize on hermit-crab hosts. Comparisons between these epibionts and shell-dwelling limpets indicate that specialization for life on shells is greatest when the host takes an active role in attracting or moving organisms to their shells. This highlights the important role of behavior in modifying the direction and extent of natural selection in host–guest associations.