Hybrid approach to classifying sky regions in natural images

Abstract

Sky is among the semantic object classes frequently seen in photographs and useful for image understanding, processing, and retrieval. We propose a novel hybrid approach to sky detection; based on color and texture classification, region extraction, and physics motivated sky signature validation. Sky can be of many different types; clear blue sky, cloudy/overcast sky, mixed sky, and twilight sky, etc. A single model cannot correctly characterize all the various types of skies due to the large difference in physics and appearance associated with different sky types. We have developed a set of physics-motivated sky models to identify clear blue-sky regions and cloudy/overcast sky regions. An exemplar-based approach is to generate the initial set of candidate sky regions. Another data-derived model is subsequently used to combine the results for different sky types to form a more complete sky map. Extensive testing using more than 3000 (randomly oriented) natural images shows that our comprehensive sky detector is able to accurately recall approximately 96% of all sky regions in the image set, with a precision of about 92%. Assuming correct image orientation, the precision on the same set of images increases to about 96%.