Virgin olive oil has a high resistance to oxidative deterioration due to both a triacylglycerol composition low in polyunsaturated fatty acids and a group of phenolic antioxidants composed mainly of polyphenols and tocopherols. Polyphenols are of greater importance to virgin olive oil stability as compared with other refined oils which are eliminated or drastically reduced during the refining process. This paper covers the main aspects related to the oxidative stability of virgin olive oil during storage as well as at the high temperatures of the main processes of food preparation, i.e., frying and baking. Differences between oxidation pathways at low and high temperature are explained and the general methods for the measurement of stability are commented on. The compounds contributing to the oxidative stability of virgin olive oils are defined with special emphasis on the antioxidative activity of phenolic compounds. Finally, the variables and parameters influencing the composition of virgin olive oils before, during and after extraction are discussed.