Know Beyond Seeing: Combining Computer Vision with Semantic Reasoning

Abstract

To date, computer vision systems are limited to extract the digital data of what the cameras see. However, the meaning of what they observe could be greatly enhanced by considering the environment and common-sense knowledge. A new approach to combine computer vision with semantic modeling has been developed. This approach extracts the knowledge from images and uses it to perform real-time reasoning according to the contextual information, events of interest and logic rules. The reasoning with image knowledge allows protecting the privacy of the users, to overcome some problems of computer vision such as occlusion and missed detections and to offer services such as people guidance and people counting. This approach is the first step to develop an all-seeing smart building that can automatically react according to its evolving information.