In this paper, we distinguish three types of symmetric games, namely, ordinary symmetric games, renaming symmetric games and name-irrelevant symmetric games, in an order of increasing broadness. Making these distinctions is meaningful even for some elementary two by two games, e.g., Battle of Sexes is renaming symmetric but not ordinary symmetric, and Matching Pennies is name-irrelevant symmetric but not renaming symmetric. In addition, some nice properties are preserved when ordinary symmetric games are extended to renaming symmetric games, e.g., when each player has two strategies, they are both exact potential games and thus pure Nash equilibria are guaranteed, which is not the case for the most general name-irrelevant symmetric games. We investigate these classes of games via the corresponding player symmetry groups, which have very rich mathematical structures. Building on the coveringness idea of Peleg et al. (1999), we provide characterizations of these player symmetry groups. We also extend the ordinially symmetric games of Osborne and Rubinstein (1994) from two to n players, and show that ordinally symmetric games with two strategies are ordinal potential games.