Permeation of compounds through membranes is important in biological and engineering processes, e.g., drug delivery through lipid bilayers, anesthetics, or chemical reactor design. Simulations at the atomic scale can provide insight in the diffusive pathways and they give estimates of the membrane permeability based on counting membrane transitions or on the inhomogeneous solubility-diffusivity model described by the Smoluchowski equation. For many permeants, permeation through a membrane is too slow to gather sufficient statistics with conventional molecular dynamics simulations, i.e., permeation is a rare event. Recent attempts to improve the description of the dynamics of such rare permeation events have been based on milestoning, which allows the study of processes at timescales beyond those achievable by straightforward molecular dynamics. The approach is not relying on an overdamped description, but, still, it uses a Markovian approximation which is only valid for small permeants that are not disruptive to the membrane structure. To overcome this fundamental limitation, we show here how replica exchange transition interface sampling (RETIS) can effectively be used on this problem by deriving an effective set of equations that relate the outcome of RETIS simulations and the permeability coefficient. In addition, we introduce two new path Monte Carlo (MC) moves specifically for permeation dynamics, that are used in combination with the ordinary path generating moves, which considerably increase the efficiency. The advantage of our method is that it gives exact results, identical to brute force molecular dynamics, but orders of magnitude faster.