A new transformation that converts a Bayesian game to a so-called <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent game (a normal-form game) is proposed. Differently from the existing transformation proposed by R. Selten that changes a Bayesian game to an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">interim</i> agent game (also a pure normal-form game), we prove that the new transformation preserves potentiality. In addition, there is a nonpotential Bayesian game whose <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent game is potential. We also prove that there is one-to-one correspondence between pure Bayesian Nash equilibria (BNE) of Bayesian games (if one exists) and pure Nash equilibria (NE) of the resulting <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent games. Then, we provide a sufficient and necessary condition for a Bayesian game to have an <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent potential game. By using these results, one can transform pure BNE seeking in Bayesian games to pure NE seeking in their <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent games [by using the potential functions of the <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent games (if one exists)], where previously pure BNE seeking in Bayesian games by using potential functions can only be done in Bayesian potential games (BPGs). Particularly, we prove for two-player games that BPGs are exactly the Bayesian games having <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">ex-ante</i> agent potential games. Furthermore, by using the semi-tensor product of matrices, a potential equation for finite Bayesian games is developed. Based on the potential equation, algorithms for verifying potentiality and for searching pure BNE in finite Bayesian games are designed. Finally, the results are applied to a routing problem with incomplete information.