Now, let’s explore the technical key steps of the Smart Retail System. The system tackles two challenging tasks simultaneously: seamless store operations automation and real-time customer movement analysis, namely- Product Classification and People Tracking.

To apply these techniques, we created a demo setup based on client needs and considering real measurements. Location mapping with Planogram Integration was used to cross-reference product movements with the store’s planogram, offering detailed insights into product placement and shelf organization. Two important aspects of smart retail are described as follows-

• Product Classification:

1. Onboarding: In this step, pictures of the products from different camera angles are captured. The products are placed on an automatic turn-table and a rotation movement is fixed to get the images. Next, it goes through a validation process to make sure the inventory data is correct. After that, unclear or irrelevant photos are filtered out using image processing algorithms. An effective and automated data handling workflow is completed when the carefully selected dataset is created. A Dockerized service is prepared so that it makes easier to collect, filter, and upload product data to AWS Cloud.
2. Semi-Automatic Annotation: This is an optimized image annotation approach. Instead of manually annotating all the images, an active learning approach is proposed for annotation. Initially, small sub-set of datasets are manually annotated and through repeated cycles of uncertainty sampling and data augmentation a model is trained. It improves the system annotation accuracy while minimizing manual work. After a few steps of training, the final model automatically annotates the entire dataset to be used for classifier training.
3. Text-Based Product Identification: In smart retail stores, reading text from product packaging is essential for accurate product identification. Cameras with relatively small field of view are setup on shelves to get the pictures of the products. Visual similarities between products can lead to confusion, but AI based techniques helps extract unique identifiers like brand names and labels. This service works in two phases: first, it builds a lookup table during product onboarding by capturing and storing textual data; second, it uses this table in real-time to identify products by matching new inputs with the stored data. This dual-phase approach ensures reliable recognition and smooth store operations.

• People Tracking:

1. The people tracking system operates using a single surveillance camera that continuously feeds video data into a deep learning-based object detection model. It employs the YOLOv8-x-pose model, which not only detects individuals in the frame but also considers their body pose, enabling more accurate and context-aware identification. This enhancement significantly improves detection in crowded or occluded environments. Each identified person is then passed on to the tracking module for further analysis. For tracking, the system utilizes the BoTSORT model, which assigns a unique ID to each detected individual by evaluating the confidence scores from the detector. BoTSORT predicts the future position of each person across frames, ensuring consistent identity tracking as they move through the store. The system logs key events such as entries, exits, and visits to specific areas like the billing counter. This data can later be used for behaviour analysis, crowd management, or optimizing store layout.