In response to environmental and climate change issues in recent decades, clean energy resources have been promoted as substitutes for fossil fuel-based dirty energy resources. While receiving great attention from environmentally conscious investors and policy-makers, the markets for the former energies could be affected by financial instability in the markets for the latter, which could lead to systemic risk and directional connectedness. In this paper, we aimed to assess potential systemic risk in such markets by employing conditional quantile- and conditional expectile-based Value-at-Risk (henceforth CoVaR and CoEVaR, respectively). Due to their definition through expected conditional asymmetric loss functions, we adopted Kabaila’s efficient method for the computation of conditional expectations and proposed a backtesting technique based on Diebold and Mariano’s test. Using Delta-CoVaR and Delta-CoEVaR, we constructed conditional tail risk networks to describe the directional tail risk connectedness. We revealed that the CoVaR and CoEVaR for almost all pairs of energy markets had the best forecast accuracy when computed using a copula exhibiting symmetric tail dependence with asymmetric and heavy-tailed margins. Compared to CoVaR, CoEVaR possessed a relatively better conditional coverage performance in several cases, complementing its advantage of being more sensitive to (extreme) loss magnitude. In addition, we found crude oil, heating oil, and sectoral clean energy indices to be highly interconnected, but the connectedness weakened following the 2015 Paris Climate Agreement. In particular, crude oil and the renewable energy sector played a consistent role as the most systemically important net receiver and net transmitter, respectively. These results provided important implications for policy-makers and investors aware of systemic risk and connectedness among dirty and clean energy markets.