Abstract
Out-of-home audience measurement aims to count and characterize the people exposed to advertising content in the physical world. While audience measurement solutions based on computer vision are of increasing interest, no commonly accepted benchmark exists to evaluate and compare their performance. In this paper, we propose the first benchmark for digital out-of-home audience measurement that evaluates the vision-based tasks of audience localization and counting, and audience demographics. The benchmark is composed of a novel, dataset captured at multiple locations and a set of performance measures. Using the benchmark, we present an in-depth comparison of eight open-source algorithms on four hardware platforms with GPU and CPU-optimized inferences and of two commercial off-the-shelf solutions for localization, count, age, and gender estimation. This benchmark and related open-source codes are available at http://ava.eecs.qmul.ac.uk.
Highlights
Digital out-of-home advertisement is rapidly growing thanks to the availability of affordable, internet-connected smart screens
Localization We evaluate the localization performance based on precision ( P ), recall ( R ), and F1-score ( F ) [4], which are defined based on true positives (TP), false positives (FP), and false negatives (FN)
Note that we only report R, and not P, for person–signage distance and occlusion as false positives cannot be unequivocally estimated when the Counting We quantify the performance of the localization algorithms for the task of people counting with the following performance measures: mean opportunity error (MOE), cumulative opportunity error (COE), and temporal cumulative opportunity error (TCOE)
Summary
Digital out-of-home advertisement is rapidly growing thanks to the availability of affordable, internet-connected smart screens. For each person j with OTS, an AVA solution is expected to produce at each time t: j, the person index (for trackers, the tracking identity consistent throughout V ); dtj , the estimated location of the face and/or body; ajt ∈ A , the estimated age; and gtj ∈ G , the estimated gender. 1.2 Performance measures We introduce a set of performance measures for assessing the accuracy of localization, counting, age and gender estimation These measures, which are concise and easy to understand by a broad community, enable the evaluation and comparison of AVA algorithms. The temporal COE (TCOE) quantifies the ability of an algorithm to count unique people with OTS over temporal segments of generic duration (e.g., 10-s duration), and it is calculated with respect to the cumulative number of unique people with OTS: Fig. 5 Sample result and evaluation for people counting.
Sign-in/Register to view highlights & summary Sign-in/Register to access full text options 
Talk to us
Join us for a 30 min session where you can share your feedback and ask us any queries you have
Schedule a callDisclaimer: All third-party content on this website/platform is and will remain the property of their respective owners and is provided on "as is" basis without any warranties, express or implied. Use of third-party content does not indicate any affiliation, sponsorship with or endorsement by them. Any references to third-party content is to identify the corresponding services and shall be considered fair use under The CopyrightLaw.
