Conditional heteroskedasticity, skewness and leverage effects are well known features of financial returns. The literature on factor models has often made assumptions that preclude the three effects to occur simultaneously. In this paper we propose a conditional heteroskedastic factor model that takes into account the presence of both the conditional skewness and leverage effects. We show that our model is robust to temporal aggregation. We then provide moment conditions that allow for inference by the Generalized Method of Moments (GMM). We conduct a Monte Carlo experiment to evaluate the finite sample properties of our estimation method. We also propose an extended Kalman filter procedure that provides a filter for both the latent factor and the volatility processes. We apply our model to a set of 24 U.K. indices, including the FTSE 350 and 23 U.K. sectorial indices. In a previous model-free investigation, we find that the daily excess returns of all these indices show strong evidence of conditional heteroskedasticity and dynamic conditional skewness and leverage effects. Our results suggest a drastic efficiency gain when both the skewness and the leverage are modeled. They also confirm, for conditionally heteroskedastic factor models for daily returns, the main findings in Harvey and Siddique (1999) (see also Jondeau and Rockinger (2003)) for univariate GARCH-type models for several data sets, namely that the inclusion of conditional skewness impacts the persistence in the conditional variance of return. We also find evidence for persistent conditional skewness. Moreover, the volatility process filtered by our model's estimates seems to better capture large shocks on returns than the one filtered by the Doz and Renault (2006) model's estimates, which ignore both skewness and leverage in data.