Flowering plant male germlines develop within anthers and undergo epigenetic reprogramming with dynamic changes in DNA methylation, chromatin modifications, and small RNAs. Profiling the epigenetic status using different technologies has substantially accumulated information on specific types of cells at different stages of male reproduction. Many epigenetically related genes involved in plant gametophyte development have been identified, and the mutation of these genes often leads to male sterility. Here, we review the recent progress on dynamic epigenetic changes during pollen mother cell differentiation, microsporogenesis, microgametogenesis, and tapetal cell development. The reported epigenetic variations between male fertile and sterile lines are summarized. We also summarize the epigenetic regulation-associated male sterility genes and discuss how epigenetic mechanisms in plant male reproduction can be further revealed.