Summary Primexine deposition is essential for the formation of pollen wall patterns and is precisely regulated by the tapetum and microspores. While tapetum‐ and/or microspore‐localized proteins are required for primexine biosynthesis, how their trafficking is established and controlled is poorly understood.In Arabidopsis thaliana, AP1σ1 and AP1σ2, two genes encoding the σ subunit of the trans‐Golgi network/early endosome (TGN/EE)‐localized ADAPTOR PROTEIN‐1 complex (AP‐1), are partially redundant for plant viability, and the loss of AP1σ1 function reduces male fertility due to defective primexine formation. Here, we investigated the role of AP‐1 in pollen wall formation.The deposition of Acyl‐CoA SYNTHETASE5 (ACOS5) and type III LIPID TRANSFER PROTEINs (LTPs) secreted from the anther tapetum, which are involved in exine formation, were impaired in ap1σ1 mutants. In addition, the microspore plasma membrane (PM) protein RUPTURED POLLEN GRAIN1 (RPG1), which regulates primexine deposition, accumulated abnormally at the TGN/EE in ap1σ1 mutants. We show that AP‐1μ recognizes the YXXΦ motif of RPG1, thereby regulating its PM abundance through endocytic trafficking, and that loss of AP1σ1 decreases the levels of other AP‐1 subunits at the TGN/EE.Our observations show that AP‐1‐mediated post‐Golgi trafficking plays a vital role in pollen wall development by regulating protein transport in tapetal cells and microspores.