Abstract
Extracting information related to weather and visual conditions at a given time and space is indispensable for scene awareness, which strongly impacts our behaviours, from simply walking in a city to riding a bike, driving a car, or autonomous drive-assistance. Despite the significance of this subject, it has still not been fully addressed by the machine intelligence relying on deep learning and computer vision to detect the multi-labels of weather and visual conditions with a unified method that can be easily used in practice. What has been achieved to-date are rather sectorial models that address a limited number of labels that do not cover the wide spectrum of weather and visual conditions. Nonetheless, weather and visual conditions are often addressed individually. In this paper, we introduce a novel framework to automatically extract this information from street-level images relying on deep learning and computer vision using a unified method without any pre-defined constraints in the processed images. A pipeline of four deep convolutional neural network (CNN) models, so-called WeatherNet, is trained, relying on residual learning using ResNet50 architecture, to extract various weather and visual conditions such as dawn/dusk, day and night for time detection, glare for lighting conditions, and clear, rainy, snowy, and foggy for weather conditions. WeatherNet shows strong performance in extracting this information from user-defined images or video streams that can be used but are not limited to autonomous vehicles and drive-assistance systems, tracking behaviours, safety-related research, or even for better understanding cities through images for policy-makers.
Highlights
Cities are complex entities by nature due to the multiple, interconnected components of their systems
There is still on-going research to cover the current limitation in addressing the weather and visual conditions simultaneously, in which addressing only one domain would not necessarily cover the dynamics of the appearance of urban scenes
While the above-mentioned models show progress in the given tasks, there are a number of knowledge gaps that need to be addressed to cover the stated subject of weather and visual classification, which are: (1) These crucial domains—weather and visual conditions—have been studied individually, ignoring the importance of understanding the dynamics and impact of one domain on the other
Summary
Cities are complex entities by nature due to the multiple, interconnected components of their systems. Features of the physical environment extracted from images, or so-called urban scenes, have great potential for analysing and modelling cities because they can contain information on a range of factors such as people and transport modes, geometric structure, land use, urban components, illumination, and weather conditions [1]. This article is concerned with the recognition of weather and visual conditions, which are two related but separate aspects of urban scenes that can be extracted in order to better understand the dynamics of the appearance of the physical environment [4]. Geo-Inf. 2019, 8, x FOR PEER REVIEW environment due to precipitation including clear, rainy, foggy, or snowy weather. They represent rcerpurceisaelnftacctrourcsiafol rfamctaonrys uforrbamnasntyuduirebsaincsltuuddiniegs tirnacnluspdoinrtg, btreahnasvpiourtr,, baenhdasvaiofeutyr,-raenladtesdafreetsye-raerlcahte[d5]
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
