Item Verification on the Smart Trolley System using Object Recognition based on the Structural Similarity Index

Abstract

Supermarket customers often encounter significant delays during checkout due to manual verification processes, wherein items are removed from the cart, handed over to the cashier, and individually scanned. To address this inefficiency, this paper introduces the design and development of an automatic payment system as part of a smart trolley solution aimed at expediting the verification process. Our goal is to develop an automatic moving trolley with smart payment devices to solve the problem. This system features a web-based payment application, which allows customers to scan their items using a barcode reader while shopping. After shopping, customers can review and confirm their items in the trolley and proceed to an exit room. Here, each item is individually verified using a camera and purchase finalization occurs. Our item verification method leverages object recognition using deep learning and similarity measurement with the structural similarity index (SSIM), which compares detected items to images stored in the supermarket's database. Our findings suggest the successful implementation of the proposed method and demonstrate that verification using the SSIM is a better alternative to traditional procedures.