Monolayers of transition metal dichalcogenides are promising materials for valleytronic applications, since they possess two individually addressable excitonic transitions at the non-equivalent $K$ and $K'$ points with different spins, selectively excitable with light of opposite circular polarization. Here, it is of crucial importance to understand the elementary processes determining the lifetime of these optically injected valley excitons. In this study, we perform microscopic calculations based on a Heisenberg equation of motion formalism to investigate the efficiency of the intervalley coupling in the presence (W based TMDCs) and absence (Mo based TMDCs) of energetically low lying momentum-dark exciton states. While we predict a valley exciton lifetime on the order of some hundreds of fs in the absence of low lying momentum-dark states we demonstrate a strong quenching of the valley lifetime in the presence of such states.