Abstract
Traditional object detection answers two questions; “what” (what the object is?) and “where” (where the object is?). “what” part of the object detection can be fine grained further i-e. “what type”, “what shape” and “what material” etc. This results in shifting of object detection task to object description paradigm. Describing object provides additional detail that enables us to understand the characteristics and attributes of the object (“plastic boat” not just boat, “glass bottle” not just bottle). This additional information can implicitly be used to gain insight about unseen objects (e.g. unknown object is “metallic”, “has wheels”), which is not possible in traditional object detection. In this paper, we present a new approach to simultaneously detect objects and infer their attributes, we call it Detectand- Describe (DaD) framework. DaD is a deep learning-based approach that extends object detection to object attribute prediction as well. We train our model on aPascal train set and evaluate our approach on aPascal test set. We achieve 97.0% in Area Under the Receiver Operating Characteristic Curve (AUC) for object attributes prediction on aPascal test set. We also show qualitative results for object attribute prediction on unseen objects, which demonstrate the effectiveness of our approach for describing unknown objects.
Highlights
Detecting objects in images has been one of the prime focus of researchers in the field of computer vision [1, 2, 3, 4]
We can infer few properties and attributes of the object (e.g. “round” bottle), we can say what is unusual about the object (e.g. “plastic” car), even if an object is unrecognizable we can still say something about it, and something about the object’s environment can be said
We show qualitative results of attribute predictions on unknown/unseen object categories by randomly selecting images from Google Image Search to demonstrate the generalization ability of our approach
Summary
Detecting objects in images has been one of the prime focus of researchers in the field of computer vision [1, 2, 3, 4]. Detection is the traditional naming task in which an object is identified by the name and localized by a bounding box in the image. Describing an object goes beyond the traditional naming task and identify few attributes and properties of the object [5, 6]. We can infer few properties and attributes of the object We can infer few properties and attributes of the object (e.g. “round” bottle), we can say what is unusual about the object (e.g. “plastic” car), even if an object is unrecognizable we can still say something about it (e.g. the unknown object “has legs”, “has head”), and something about the object’s environment can be said (e.g. train “has leaf” means train is in trees).
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a call
