Skip to main content

Barcode Less Items Recognition (BLIR) on Lane POS - Integration Guide

Solution Description​

This document outlines the integration guide for integrating with Edgify agent via REST APIs. The Lane POS UI application (“UI Application”) interacts with the Edgify Agent (“Agent”) via 2 main REST APIs: Capture & Predict and Put Sample (see below). The Edgify Agent can be installed on a Windows device, in which it will run as a service or on Linux devices in a Docker container. The agent enables Lane POS devices to execute real-time predictions and obtain the certainty of these predictions. The agent is capable of detecting false selections and initiating intervention procedures via the POS screen and attendant application.

Video Examples You can see here an example of integrating with our software:

  • Lane POS integration for Shufersal

API Flow​

The calls need to be authenticated using “Digest Authentication” *Username and Password should be in a config file and will change between customers.

Capture & Predict​​

POST http://localhost:8090/api/v2/capture-and-predict

The Agent captures an image and runs a prediction on it, (Image is in base64 text). In The Body you will need to add captureTrigger and family.

Body Example:​


{
 "captureTrigger": "Picklist selection+scale stable",
 "family": "Bakery"
}

Parameters Explanation:​​

  • “captureTrigger” informs Edgify what user experience triggered the call to this Edgify prediction call.
  • “family” parameter - the parameter informs Edgify from what item’s assortment the prediction will probably be. This value can come from the name of the menu in the POS application. Example family values:​
    • Fruits
    • Vegetables
    • Bakery
    • Cases
    • OrganicVegetables
    • OrganicFruits
    • ScannedBarcodeItem

The call provides an output with: sampleId image certain(true/false) the predictions and confidence

Examples:​​

In the following example: The agent predicts the item is SKU # “1000000002940” with Confidence of 0.4597 and 2 other SKUs 1000000003012 with confidence of 0.2153 & 1000000000029 with confidence of 0.211, and we were not certain (false) what is the item For unknown items, the agent will return a prediction object with “Unknown” in the prediction array. In the following example: The agent predicts “Unknown”.

“Certain” field​​

The Certain parameter is related to a use case where you would want to use "AUTO BUY" and to add the item automatically to the shopping list. Auto Buy is a situation where you would want an automatic flow to happen when the prediction is certain above a defined threshold. For example, whenever Edgify predicts an item by above 99% add the item to the shopping list. In the Edgify agent configuration there is a parameter to configure what would be the threshold to output certain:true (Example in above previous Print Screen).

Put Sample​​

PUT http://localhost:8090/api/v2/samples/{sample_id}

Description :​

Saves the sampleId and Image with the chosen label, after the customer has chosen the product.​ Same as the Capture & Predict API, The call needs to be authenticated using “Digest Authentication” Username and Password should be in a config file and will change between customers Each prediction has a unique sample_id, use that id in the api request In the request body.

  • “Label” - Is the Product PLU. This will be part of the model and will be recommended in the output prediction
  • “labelTrigger”- (previously was called “Source”) is how the customer decided to choose/confirm the label selection. Example values:
    • RecommendationStrip
    • RegularMenuSelection
    • Auto Buy
  • "captureTrigger" - (Optional) Should be the value that was passed in the captureTrigger parameter for the prediction object which this ground truth sample is related to.
  • "family" - Should be the name of the item's menu group. Example values: Fruits / OrganicFruits Vegetables / OrganicVegetables Bakery Cases
  • "category" - (optional) Should be the catalog hierarchy for which this SKU belongs within the retailer catalog DB. Example: “Bakery” if the SKU is a baked item. It's supposed to come based on the real relation of the SKU within the retailer’s products catalog (if possible).
  • "groupId" - (optional) for other Edgify use cases
  • "imgSource" - (optional) the name of the device capturing the image
  • "Image"- the image itself (a Base64-encoded jpeg image received from the capture-and-predict call earlier)
  • “price"- the product price per the weighing units (i.e- Kg/Ibs)
  • "weight"- the weight of the item that was received after the scale was stable.
  • "count"- relevant for items whose price is by quantity and not by weight.
  • "totalPrice"- the total price that the shopper sees on the POS screen after the scale is stable and the item is selected.

Put Sample Response - Match Strength​​

The PUT Sample call returns a matchStrength value indicating whether this sample does not match the label provided. Response Object Example:​

{
   "matchStrength": "alert-non-match"
}

The possible values for the matchStrength are as follows:

  • match - The label provided matches the image
  • alert-non-match - Edgify is confident that the label does not match, might prompt an alert to be displayed on the screen and potentially trigger attendant verification.
  • unknown - Edgify was not able to confirm or alert

POST events - Upon Intervention Complete​

POST http://localhost:8090/api/v2/events

Description:​​

The following endpoints should be called upon the shopper's selection. They enable the system to measure the impact of the intervention.

POST /event with "non-match-continue​"​

POST http://localhost:8090/api/v2/events
{ 
    "name": "non-match-continue" , "sampleId": "UUID",
}

POST /event with "non-match-startover​"​

POST http://localhost:8090/api/v2/events
{ 
    "name": "non-match-startover" , "sampleId": "UUID", 
}

The following endpoints should be called upon the attendant's selection. They enable the system to measure the impact of the second level of intervention.

POST /event with "attendant-rejected"​

The attendant rejected the alert

POST http://localhost:8090/api/v2/events
{ 
    "name": "attendant-rejected" , "sampleId": "UUID",
}

Delete Sample​

Delete Sample​- Store manager passes the card and remove the item from the shopping list

DELETE http://localhost:8090/api/v2/samples/{sample_id}

Description :​

Deletes the passed Sample from the Edgify system.

  • For specialized use cases. Please consult your Customer Success Team.

Important Logs​

Make sure you write to the POS application log upon each call to Edgify's APIs. Include timestamp and the meta data sent and received.

Documentation​

Please provide basic documentation or a short video showing how to use / navigate the integration and staff logins or backend menus included in your application software.

Testing Integration of Self CheckOut​

After integration we do recommend jumping on a call with Edgify Sales Engineer (SE) to test the integration.

A few use cases should be tested after integration:

  • Edgify returns 1 item
  • Edgify returns multiple items
  • Edgify returns “Unknown”
  • Edgify Certain: true - (If this flow was implemented)
  • Edgify gets GT with the correct data.
  • Edgify is down (no response)
  • Toggle Silent/Active Mode

Note: For developing & testing purposes Edgify can change the configuration to provide any of the above flows every time by changing the agent parameters.

Data Collection Flow​

Edgify recommends you Implement a flow for collecting product image samples and SKUs for new Products In order to be able to add new SKUs to the system without having to wait for customers actually buying each item 500 times. You might want to add to your client software the capability to capture images of new items faster. Hence, it's important that the client software will implement a feature for capturing multiple images and ground truth rapidly.