To distinguish colors, the nervous system must compare the activity of distinct subtypes of photoreceptors that are maximally sensitive to different portions of the light spectrum. In vertebrates, a variety of adaptations have arisen to refine the spectral sensitivity of cone photoreceptors and improve color vision. In this review article, we focus on one such adaptation, the oil droplet, a unique optical organelle found within the inner segment of cone photoreceptors of a diverse array of vertebrate species, from fish to mammals. These droplets, which consist of neutral lipids and carotenoid pigments, are interposed in the path of light through the photoreceptor and modify the intensity and spectrum of light reaching the photosensitive outer segment. In the course of evolution, the optical function of oil droplets has been fine-tuned through changes in carotenoid content. Species active in dim light reduce or eliminate carotenoids to enhance sensitivity, whereas species active in bright light precisely modulate carotenoid double bond conjugation and concentration among cone subtypes to optimize color discrimination and color constancy. Cone oil droplets have sparked the curiosity of vision scientists for more than a century. Accordingly, we begin by briefly reviewing the history of research on oil droplets. We then discuss what is known about the developmental origins of oil droplets. Next, we describe recent advances in understanding the function of oil droplets based on biochemical and optical analyses. Finally, we survey the occurrence and properties of oil droplets across the diversity of vertebrate species and discuss what these patterns indicate about the evolutionary history and function of this intriguing organelle.