7.0 Accuracy Audit Workflow

FootfallCam will perform an accuracy audit to verify the integrity of the data generated by each FootfallCam device when tuning is requested upon device installation. There are 2 main parts in accuracy audit, tuning and certification. The accuracy audit for each device is expected to be completed within 3 working days under ideal circumstances.

Tuning is settings adjustments based on the device's live view while certification is the cross-checking process on manual count with the system count based on short pre-recorded video clip(s). This is to ensure that the accuracy is at least 90% with enough sample size to confirm on the accuracy. Once the accuracy audit is completed, an accuracy audit report will be generated and sent to the user.

 

7.1 Accuracy Audit

Accuracy Audit indicates the readiness of the device's data. Before user starts checking on the data of a device or a site, users shall check on the accuracy audit stage first to make sure the data is ready to be used.

 

 

7.1.1 Accuracy Audit Stages

Users can view the stage of the accuracy audit for each devices from the Device Management & Health Check page by following the steps below:

STEP 1 - Access to FootfallCam Analytic Manager V8™: http://portal.footfallcam.com/Account/Login.

STEP 2 - Click on Setting > Devices to access the Device Management & Health Check page.

STEP 3 - Check on the 'Accuracy Audit' column.

The stage of accuracy audit can be also be viewed in Device Page after clicking on the Edit of the device of interest:

The following are the accuracy audit stages for devices:

Accuracy Audit Stage	Descriptions
#1 New	The device is newly allocated which requires usage and configuration confirmation.
#1 Ready to Tune	The device has been configured and is ready for tuning by FootfallCam.
#2 First Line Tuning	The tuning process of the device has been started and is expected to be completed within 3 working days under normal circumstances.
#2 Second Line Tuning	There is minor difficulty in tuning the device which requires FootfallCam tuning specialist to look further into the issue. Please expect a minor delay in the accuracy audit completion date. (optional stage)
#2 Ticket Raised	There is environmental constraint which restrain the accuracy audit process of the device from being able to be proceeded which requires action from client. Please refer to the relative support tickets for more info. (optional stage)
#3 Tuned	The device's settings have been refined and the data is ready to be used. In the meanwhile, FootfallCam will continue to collect sufficient sample size in order to prove the counting accuracy with video evidence.
#4 Completed	The device has successfully completed the accuracy audit with statistically significant sample size. Devices are expected to be completing accuracy audit within 7 working days under normal circumstances. Please click into the link to view the accuracy audit report.
No Need to Tune	Device's accuracy audit is put on hold due to testing purpose or constraints not being able to resolved, which would result in inaccurate counting for the device.
Reseller to Tune	The accuracy audit of the device will be carried out by FootfallCam Partner.

The data of a device is ready to be used only when its accuracy audit stage is either '#3 Tuned' or '#4 Completed'.

As for the site data to be ready, all the active devices at the site must either be '#3 Tuned' or '#4 Completed'.

 

7.1.2 Accuracy Audit Detail Process

Accuracy Audit
Completed By	FootfallCam tuning personnel
Process Duration	Within fourteen (14) working days 3 working days - Device tuning completed and device counting function as per standard. 4 - 14 working days - Collect sample size and generate Accuracy Audit Report. (Info: Process duration might be increased due to lack of sample size or environment constraints.)
Scope of Work	First-time free verification for new device ONLY. Define size of tracking zone. Define counting line configuration. Define masking zone to filter inanimate objects. Schedule video recordings for manual verification. Manual video verification. Generate Accuracy Audit Report.
Result	Over 90% counting accuracy with at least 20 sample size. Receive Accuracy Audit Report via email.

 

7.1.3 Download Accuracy Audit Report

Accuracy Audit Report for devices can be obtained from the Accuracy Audit section in the Device Page (refer 7.1.1 to be redirected to Device Page). Click on View Report to be redirected to the corresponding accuracy report.

 

7.2 Self Accuracy Auditing

Users can perform self accuracy auditing whenever the counting accuracy is in doubt. This can be done in a few steps:

1. Create new recording (refer to 7.2.1 for more details)
2. Play video when the file is available
3. Cross-check manual counting (refer 7.2.2 more more details) with system counting
4. Generate accuracy audit report (optional, only required when users would like to keep a copy of the report as a reference for the counting accuracy based on the new videos; may refer to 7.2.3 for more details)

 

7.2.1 Create New Recording

In order to schedule videos, follow the steps below:

STEP 1 - In the Device Page (refer 7.1.1 to be redirected to Device Page), scroll to the bottom and navigate to Accuracy Audit tab, click on the Schedule Video button to be redirected to the Verifying Video page. 

 

STEP 2 - Click on Create New Recording and a Schedule Recording pop-up will be shown.

STEP 3 - Complete the process by entering all the required fields and click on Set Schedule button.

Field	Description
1. Site Operating Hour	Operating hour for the branch. (Info: For more information on Site Operating Hour, please refer to Section 4.5 Manage Site Detail.)
2. Counter Local Date Time	Current date time for the device.
3. Recording Type	Select the type of recording. (Choose 'Normal' for self auditing purpose.)
4. Recording Date	Select the date to record.
5. Recording Time	Select the time and duration to record.
6. Add Schedule	Add another date to record
7. Remove Schedule	Remove schedule date.

 

7.2.2 Perform Manual Counting

STEP 1 - Go to Verifying Video page (may refer to the steps in 7.2.1 Create New Recording).

STEP 2 - Select any available video and click on Verify to perform manual count by referring the guideline as below:

Visitor must go through the start-end of tracking zone (Red) and pass-by the In line (Green) then only will be counted as IN +1.	If two visitors go through the start-end of tracking zone (Red) at the same time and pass-by the Out line (Blue), both will be counted and OUT will be showing +2.	If two visitors go through the start-end of tracking zone (Red) at the same time and pass-by the In line (Green) or Out line (Blue), both will be counted as IN +1 and OUT +1 respectively.
Children (height < 1.3m) and inanimate objects will NOT be counted.	Visitor that make multiple U-turn within start-end of tracking zone (Red) and pass-by the In line (Green) will be counted as IN +1 only when he exits the tracking zone.	Visitor who go through the start-end of tracking zone (Red) and pass-by the In line (Green) but make a U-turn after exiting the tracking zone and pass-by the Out line (Blue) again will be counted as IN +1 and OUT +1.
Visitor (Head) that walk outside of start-end of tracking zone (Red) will NOT be counted.	Visitor that go through the start-end of tracking zone (Red) only without passing-by the In line (Green) or Out line (Blue) will NOT be counted.	Visitor that make U-turn within start-end of tracking zone (Red) will NOT be counted.

STEP 3 - Perform manual count with Keyboard Shortcut.

STEP 4 - Complete the process and click on Complete Verification Study button.

Item	Description
1. Video Recorded	The footage recorded to perform verification study. Red Line (Mandatory): Tracking zone. Blue Line (Mandatory): OUT line. Green Line (Mandatory): IN line. Yellow Line (Optional): Staff exclusion zone.
2. Playback Speed	To adjust the playback speed of the video.
3. Total System Count	To show the total visitor IN and visitor OUT counted in the video recorded.
4. System Count	To show the live counting when playback through the video recorded.
5. Manual Count	UP ARROW: Manual Count as OUT. DOWN ARROW: Manual Count as IN.
6. Add Schedule	To show the accuracy of counting based on System Count vs. Manual Count.
7. Remove Schedule	To show the manual count triggered with timestamp of the video recorded.

 

7.2.3 Delete Recording

STEP 1 - Go to Verifying Video page (may refer to the Step 1 to 4 in 7.2.1 Create New Recording).

STEP 2 - Check the box on the left for the video you wish to delete and click on Delete Video.

 

7.3 Site & Device Profile

From Site Management & Health Check page, user should be able to view a column for 'Site Profile'.

The tags here indicates the site readiness, site usage, and known but unsolved constraint issues for the respective sites. Known issues will first be informed to users through support tickets by FootfallCam, if there is no reply from users or the issues are not solved, then those tags will be added to the respective sites.

From Device Management & Health Check page, user should also notice there is a column for 'Device Profile'.

The tags here indicates some nature of the device itself either on the device readiness, other usages or some known but unsolved constraint issues for each device. These tags can only be added by FootfallCam personnel.

Those tags requiring action by users will then be explained in more details in section 7.3.2.

 

7.3.1 Ideal Environment

When a device is installed under an ideal environment for counting, it will not be tagged with constraint issues and will be achieve an accuracy of at least 90%.

An ideal device environment consists of the following criteria:

• internet connection available
• correct device model according to the mounted height
• sufficient lightning condition
• installed at a flat ceiling with flat walking area
• live view is free of all sorts of blockage
• no queuing within tracking zone
• sufficient tracking zone within live view (refer to 7.3.1.1 Live View Checking for more checking details)
• store staff is not hovering within tracking zone

 

7.3.1.1 Live View Checking

Where to Check

A device's latest live view can be checked by 2 methods:

1. Checking current live view (recommended): Accessing the device based on the steps in 3.0 Accessing the Device under installation guide.
2. Check yesterday's live view (will not know if there is any change in live view today): Check from Data Integrity Report of the corresponding site. In order to check Data Integrity Report, follow the steps below:

STEP 1 - Login to FootfallCam Analytic Manager V8™.

STEP 2 - Click on Setting > Site Management to access the page.

STEP 3 - Click 'Edit' of the site of interest to enter the Site Page.

STEP 4 - From site page, click 'View' at the Data Integrity Report section to open the report.

STEP 5 - In the data integrity report, search for the device and there will be a live view section for the device at the last part for the device.
Last Verified Live View is showing the live view being referred during accuracy audit process while the rest are the live view on the last day for each month.

What to Check

1. Ensure no objects/ decorations is blocking the walking paths of visitors
2. Lightning condition is good enough for the walking path to be seen clearly from the live view
3. Both lens of the device is clear
4. Swinging door is only seen less than 30% in the live view when it is swinging inward (best if it is not in live view at all)
5. If the device is for entrance counting, ensure the whole entrance is covered based on the recommended device count by using FootfallCam calculator

 

7.3.2 Constraints Affecting Counting Accuracy

The following are the constraints where the counting accuracy of a device might be affected when the issues are not solved and their corresponding action plans:

Constraints	Descriptions	Effect	Action Plan
Incorrect Degree Lens Device Used	Lens type of the device used is not recommended by FootfallCam	Device will not be able to track and count visitors properly, leading to miscount	Check the ceiling height of the location where the device is installed is requiring which lens type by referring to FootfallCam calculator
Low Ceiling	Mounting height is lower than 2.5m	The head of the visitors will be way too large being viewed in the live view and tracking distance is not sufficient for counting purpose, leading to miscount	Reposition the device to a higher position
High Ceiling	Mounting height is higher than 7m	The device will not be able to track the visitors, leading to miscount and thus missing data	Use a wall bracket or a drop pole to position the device to a lower position
Live View Changed	Latest live view is different from the one during accuracy audit	Counting is probably not accurate anymore due to new store layout or device position	Raise a support ticket to FootfallCam requesting for reverification due to new store layout or new position
Live View Blocked	Walking path in live view is blocked by object/ decoration	Visitor cannot be tracked with a complete and sufficient walking distance to be counted, leading to miscounts	Remove the blocking object/ decoration from the live view
Blur Lens	Lens of the device is blurred usually with dirt or fingerprints	Device will not be able to track visitors over the blurred area, leading to miscount if the blurred area is covering the walking path	Gently wipe the device's lens with a dry cloth and check the live view again to make sure it is now clean
Dark Environment	Live view is too dark and visitors are barely visible	Device can hardly track visitors, leading to miscount and might be triggering false counting when the device is forced to be counting in the dark	Enhance the lightning condition of the environment where the device is being installed
Limited Coverage	Entrance is not fully being covered by all the devices at the site	Parts of the entrance is not being covered by the devices at the site, leading to miscount and often to high site discrepancy	Install the suggested number of devices according to the entrance width by referring to FootfallCam calculator
Swinging Door	When the door is being swing inwards, it is covering more than 30% of the live view	The door will be triggering false counting and might lead to miscount of visitors at the same time	Reposition the device to be further away from the door so that the door is not covering more than 30% of the live view when it swings inward
Staircase/ Slope Path/ Installed at a Tilted Angle	The tracking area in the live view is not a flat surface	When visitors walk across the live view, their heights will not be consistent, leading to miscount	Reposition the device to an area with sufficient flat walking area (can consult FootfallCam though support ticket prior to the repositioning)
Faulty Device	Device has been confirmed faulty by FootfallCam	Device is not able to count accurately anymore	Proceed with the steps stated as in the relative RMA ticket (device within warranty) or replace the device with a new one
Limited Sample Size	Device does not have sufficient sample size for accuracy audit	The counting accuracy might be lower as there are no enough sample size to validate the counting accuracy during accuracy audit	Arrange a walk test with FootfallCam by raising a support ticket to FootfallCam informing the desired date and time for the walk test 2 working days prior to the desired date
Limited Tracking Area	The walking area within live view is not sufficient	There is insufficient tracking distance for the device, leading to miscount	Reposition the device to an area with sufficient tracking area
Staff / Shopper Activity Triggered Counts	Device is installed at a position where staff/ visitors are constantly hovering around	Staff/ visitors might be counted multiple times when they constantly passing across the tracking lines and out of the tracking zone	Advice staff to stay out from the tracking zone
Unclear Entrance	Multiple entrances/ no entrance can be viewed from the live view	Wrong tracking zone might be configured, leading to false counting	Inform FootfallCam through support ticket upon the device installation on either the device is for hallway counting or on which entrance should be covered

Raise a support ticket to inform FootfallCam after the recommended action plan has been taken, either solving the issue or not, to let FootfallCam knows the next step to proceed.

(Info: For more information on Raising a Support Ticket, please refer to 24.3 Raising an Issue.)

 

7.4 Other Minor Configurations

The standard accuracy audit of a FootfallCam device is based on the purpose of IN OUT counting of visitors with the height of 1.3m and above. However, there are some other additional configurations which can be applied upon request from users. Among those there are some minor configurations which can be requested through support tickets, as listed in the following sections:

• 7.4.1 Only IN Count Required
• 7.4.2 Only OUT Count Required
• 7.4.3 Kids Counting
• 7.4.4 Exclusion Line
• 7.4.5 Staff Exclusion Tag
• 7.4.6 Staff Exclusion Button
• 7.4.7 GPIO

As for those with separate reports or other types of counting, their details will be in other chapters:

• Chapter 16: Heat Map Configuration
• Chapter 17: Live Occupancy Configuration
• Chapter 18: Sales Conversion Configuration
• Chapter 19: Queue Counting Configuration
• Chapter 20: Zone Counting Configuration
• Chapter 21: Area Counting Configuration
• Chapter 22: Bus Solution Configuration
• Chapter 23: Electronic Shelf Label (ESL) Configuration

(Info: For more information on Raising a Support Ticket, please refer to 24.3 Raising an Issue.)

 

7.4.1 Only IN Count Required

• Description: Only the IN count of a site/ device can be viewed from reports.
• Usage: This can be used when users are only interested in the IN counting of a site or when a device is installed at the entrance of the site. Upon the request for this configuration, FootfallCam and users will not receive any false alarms on IN OUT discrepancy and it is not necessary for users to install devices at the exits as well.
• Warning Message: OUT count will be automatically excluded from all reports.
• Input from Users: The device serial of which the device is installed at the entrance or site name of which not all entrances of the site are installed with FootfallCam device.

 

7.4.2 Only OUT Count Required

• Description: Only the OUT count of a device can be viewed from reports.
• Usage: This can be used when users are only interested in the OUT counting of a device which is installed at the exit of the site.
• Warning Message: IN count will be automatically excluded from all reports.
• Input from Users: The device serial of which the device is installed at the exit of the site.

 

7.4.3 Kids Counting

• Description: The minimum tracking height of all devices at a site will be adjusted to a minimum of 1.1m instead of 1.3m (standard).
• Usage: It is used to include the counting of visitors with the height of 1.1m to 1.3m, especially kids.
• Warning Messages:
  1. FootfallCam would require more days to tune a device for this usage.
  2. Once this is applied, other objects at the similar height might be wrongly counted as visitors (e.g. trolleys, baskets, strollers).
  3. Kids walking closely to an adult might be miscounted.
  4. Most probably miscount would happen. Therefore, FootfallCam will not monitor on the site discrepancy anymore after this configuration is applied to any device at a site.
• Input from Users: The site name of which users wish to apply this configuration

 

7.4.4 Exclusion Line

• Description: A line in cyan colour will be drawn across a specific area stated by users (usually drawn at a very small area at the right/ left most of an entrance with the least visitors passing through).
• Usage: It is used to exclude the counting of visitors across the exclusion line, especially used to exclude staffs from being counted.
• Warning Message: There is no guarantee that ordinary visitors will not enter/ exit across that line, so visitors might also be excluded from counting and FootfallCam will no longer monitor on the site's IN OUT discrepancy once any of the device at the site is applied with exclusion line.
• Input from Users: The device serial of which users wish to have exclusion line and screenshot of device's live view indicating where the line should be drawn.

 

7.4.5 Staff Exclusion Tag

• Description: Devices with this configuration will exclude the counting of whom wearing the specially designed tag by FootfallCam.
• Usage: It is used to exclude staffs from being counted as ordinary visitors as staffs may enter/ exit the site multiple times per day.
• Additional Kit: A specially designed tag by FootfallCam
• Warning Messages: 

  1. Walk test of staffs wearing the tag is required.

  2. Special accuracy audit is required which takes a longer time than usual devices.

  3. Not applicable for devices installed at low ceiling (usually those lower than 2.5m).

• Input from Users: Number of tags required and site name of which users wish to use staff exclusion tag.

 

7.4.6 Staff Exclusion Button

• Description: A zone will be drawn at an area specified by the users with staff exclusion button installed.
• Usage: It is used to exclude the counting of visitors in a specific zone who pass through the zone and clicked on the button (usually used to exclude staffs).
• Additional Kit: Staff Exclusion Wired Button
• Warning Message: Anyone who clicked on the button and passed through the zone will be excluded from counting and there is a possibility ordinary visitor may press on the button too, meaning that those will be excluded as well.
• Input from Users: The device serial of which users which to have staff exclusion zone and the screenshot of the device's live view indicating the area which users which to cover.