Graph-based non-maximal suppression for detecting products on the rack

Abstract

Abstract Identification of stacked retail products from the images of racks of supermarket is a challenging computer vision problem. Regions with convolutional neural network features (R-CNN) generate (mostly overlapped) region proposals around the products on the rack. Subsequently, the region proposals are classified using a convolutional neural network. In the end, R-CNN implements a greedy non-maximal suppression (greedy-NMS) for disambiguating the overlapping proposals. Greedy-NMS discards the proposals (with lower classification scores) that are overlapped with the proposal with higher classification score. This greedy approach often eliminates the (geometrically) better fitted region proposals with (marginally) lower classification scores. This paper introduces a novel graph-based non-maximal suppression (G-NMS) that removes this critical bottleneck of greedy-NMS by looking not only at the classification scores but also at the product classes of the overlapping region proposals. G-NMS first determines the potential confidence scores (pc-scores) of the region proposals by defining the groups of overlapping regions. Subsequently, a directed acyclic graph (DAG) is strategically constructed with the proposals utilizing their pc-scores and overlapping groups. Eventually the maximum weighted path of the DAG provides the products that are present in the rack. The results of our extensive experiments confirm that the proposed scheme is better up to around 7% on one large In-house and three benchmark datasets of retail products. Additionally, the efficacy of our proposed GNMS is also analyzed on four benchmark datasets for detecting generic objects.