Changes

== Soft Launch ==

== Soft Launch ==

The soft launch comprised six inspectors from Quebec, Ontario, and British Columbia regions to use the application to scan cosmetic products in their duties as inspectors and specifically if they went onsite to international mail centres from December 2022 to February 2023.

The soft launch was conducted with six inspectors from Quebec, Ontario, and British Columbia regions to use the application to scan cosmetic products in their duties as inspectors and specifically if they went onsite to international mail centers from December 2022 to February 2023. The purpose was to gain feedback on user experience, and the efficacy and speed of the app in identifying ingredients in real life scenarios.

=== Overall Feedback ===

=== Overall Feedback ===

The soft launch ended with approximately 63 scans of cosmetic products between the six participants.

The soft launch ended with approximately 63 scans of cosmetic products between the six participants. In a follow up survey, six participants indicated that the application "made their jobs easier". Furthermore, several commented on the time saving benefits of using the 8ball application. 

The six participants conducted a follow-up survey where they indicated that the 8ball APP “made their job easier”

During testing, there were comments and concerns about the speed of the application. Further testing of the 8ball application demonstrated that the speed issue was a result of the application's inability to retrieve submission within a reasonable time. It was found that the internet and cellular data speed were not the main factors causing the delays in processing speed. In response to this, the app was updated, which significantly improved the speed.

*The thought of being able to use an app to help identify hotlist ingredients for cosmetic products without having to go to the Hotlist website or shared drives and search for it saves a lot of time”, and

Twenty submissions were randomly selected for analysis to determine the time saved by using the application compared to manually typing and searching for ingredients. Figure 1 illustrates the results of the analysis after outliers were removed. Overall the application processing time (2.9 minutes) is faster on average compared to manual process.

*The first impression of Soft Ball was very good as it is a useful tool which can make work easier for Inspectors”.  

[[File:Figure.png|center|thumb|578x578px|Time saving analysis of 20 submissions during soft launch after removal of outliers. The average time saved is indicated by the red bar. The green bar demonstrates submissions where the application had lower processing time, while the yellow bars represent higher application processing time. ]]

During testing, there were also concerns and comments raised about the speed of the application. Five of the six participants reported bugs were related to the speed or completion of a submitted scan. This was brought up with the developers after the soft launch was completed to correct the bug.

 

In response to the bugs reported regarding the APP not being able to provide submission results in a timely manner, testers from ON, PR, BC and NCR regions tested on both mobile and desktop version of the 8ball APP with developers on March 10, 2023. Test results demonstrated the application was unable to retrieve submission within a reasonable time, causing back-end resubmissions and eventual crashes. During the test, it was proven the internet and cellular data speed were not the main factors causing the delays in processing speed of the APP. This indicator provides a level of confidence that the application can be supported in warehouses and mail centres where internet connectivity may be reduced.

 

The APP was updated on March 21, 2023, after a planned meeting with the developers and the soft launch team. After the update, the APP showed significant speed improvements in the cloud side “Submission to assessed” analysis which can help provide more use of the APP during onsite inspections. Further testing is required to help determine the extent of the update.

 

To determine the time saved using the APP compared to manually typing and searching for ingredients, 20 submissions were selected from the 63 scans during the soft launch for analysis. Students from Ontario region were asked to randomly select 20 submissions and compare their typing speed to the APP’s processing speed. Figure 1 illustrates the average processing time for the APP was 200.11 seconds (3.3 minutes) while the manual process time was on average 262.96 seconds (4.4 minutes). Outliers where the APP was taking an exceptionally long time to process is highlighted in red font in Figure 1. One potential reason for the outliers is the spin up time for the cloud server for the application. Each outlier was the first scan of each batch the tester completed. Further testing can show if this hypothesis is correct, or another issue is the cause for the outlier. The bolded fonts under the ‘Time Difference (APP-Manual in sec)’ column reflect the submission where the APP processing time was longer than manual processing.