Project 8Ball
Currently, Consumer Product Safety (CPS) inspectors must verify the compliance of consumer products and cosmetics at various locations such as land border sites and at locations operated by their regulated party. In addition to verifying compliance as per the various regulations under the Canada Consumer Product Safety Act (CCPSA) and Food and Drugs Act (FDA), the program also collects information for documentation and surveillance purposes. Enforcement is typically taken on-site, however the current compliance verification process is lengthy and inefficient due to various steps involved. During this process inspectors are expected to review the past compliance history of the products and the regulated party, information that is typically stored in varying sources such as internal Health Canada databases and the internet.
Project 8Ball was initiated through collaboration between consumer product safety inspectors across Canada and colleagues from Health Canada’s Triage and Surveillance division (TSD) and National Capitol Region (NCR). Inspired by the popular Magic 8 Ball fortune telling ball, Project 8Ball will stand as a resource for Government of Canada employees to enter information and “magically” receive an instant answer of advice. Project 8Ball was created to leverage the technology developed in Project Cyclops to develop a user-friendly mobile application that will be used as a support tool for regulatory decisions on at Canadian border locations. The scope of Project Cyclops was limited to focusing solely on natural health products (NHP). Project 8Ball was able to utilize existing technology to expand the scope of Project Cyclops to focus on cosmetics. This tool will allow Government of Canada employees to have relevant and tailored information at their fingertips (see icon mock-ups in design process). The mobile application will include automatic label reference tool to detect cosmetic ingredients from the Restricted and Prohibited Hotlist to assist inspectors when making admissibility decisions at the border. As well, have capabilities to generate reports that may be used by inspectors for detailed documentation, compliance promotion, outreach, education, and future surveillance.
Development- Project Cyclops
Phase 1: Initiation & Stakeholder Engagement - Completed
During the initiation and stakeholder engagement phase, we will be finalizing work plans and obtaining approvals. We will then be identifying the requirements for product labels for NHPs as per the Food and Drug Act under natural health product regulations and guidance documents. A collaboration with ROEB programs will be required to apply data from inspection processes requiring label reviews. This will include the health program appliance to understand the inspection process, as well as the information recorded for compliance verifications or the compliance-monitoring project that would require label review, and the Border Integrity Program to understand the border inspection process, with the labeling review undertaken for NHPs reviewed at the border.
There will be a co-creation with BREAD, BSFO and IMSD to ensure the project aligns with the departmental approach for potential scalability.
Phase 2: Business Analysis and Functional Requirements (The Discovery Phase) - Completed
During the business analysis and Functional requirements phase, the business processes will be mapped for label reviews within ROEB, including the output required for the completion of the inspection specific to each group. Functional requirements will be identified; those of which would be required for the MVP for use in the field. During this process BRD and SRD will be developed and reviewed.
Phase 3: Development (Agile process)
During this phase, economies of speed will be used to procure professional service(s) that are able to identify the model required to test feasibility of a solution that is able to conduct functions. These functions include text scanning, with image and video analysis capabilities of 3D objects having varying predictive capabilities based on angular projections, and automation of real-time review and assessment of product labels using artificial intelligence enabled by predictive insight, with information from acute and legacy data. Proof of concept will be developed, which will be used to demonstrate features required for an MVP.
Next Steps – Project 8Ball
Stream 1
The scope of the 8Ball Project in Stream 1 is to develop the model, or a preliminary proof of concept that has demonstrable features that would be required for a horizontally designed Minimum Viable Product (MVP). IMSD will play a pivotal role in providing support and guidance to the vendor and ensure backend compatibility to Health Canada infrastructure. By using technology developed in Project Cyclops, the feasibility of the solution will be demonstrated. The MVP should be able to review product labels for a Cosmetic product during an on-site inspection with a random sampling of products, in a real inspection setting.
The 8ball mobile application tool will provide digital photos, translations and the conversion of images, printed text or handwritten into machine encoded text (Optical Character Recognition (OCR)) in real time. Inspectors will be able to capture images of a product and with OCR the tool will cross-reference it to the frequently updated Hotlist Ingredient List (see images in design process). The mobile tool will provide live translations of English, French, and other languages to verify compliance against labelling requirements and the Hotlist Ingredient List. Consumer Product Safety (CPS) has at least 16 regulations that include specific labelling requirements with pre-defined text specified. Labels will be scanned, and the existing text would be compared to the regulatory requirements. The process output would identify any existing deficiencies (view process flow below). Additionally, pending co-operation of Canada Border Services Agency (CBSA) officers, the application will be offered to the CBSA officers to be used on a voluntary basis to help focus on providing improved referrals to Health Canada.
Throughout the development process of the 8Ball application, a list of non-negotiable requirements for the final product was established. The application should have features that include:
- Accessiblity on mobile devices and smart tablets
- Analysis runtime and report output less than one (1) minute
- Ability to analyze both French and English labels on varying surface types
- Ability to recognize languages other than French and English and translate into French or English
- Ability to categorize cosmetics products in order to identify specific, applicable ingredient restrictions, labelling requirements and usages
- Capacity to have multiple users (Maximum 20) using the application at the same time
- Ability to harness active learning to continuously improve functions through continued use
- Ability to run through a Government of Canada Azure Cloud
- Refreshes information utilized from the APIs daily
Stream 2
In the final stage of the project, we will assess if the MVP is appropriate for enterprise release. If appropriate, the project would seek to secure funding to scale accordingly via the Investment Plan process. The project will take an inclusive approach with both inputs and outputs, critical to perpetuate innovation. The tool will provide a voluntary sign-up-based platform for Canadians to access outreach information such as recalls and alerts of consumer products and cosmetics, consultations, reports, etc.
Design Process
The 8Ball design process strives to combine optimized functionality, usability and creativity.
Logo
Initial 8Ball logo sketches (see below) have been designed to combine several elements that the 8Ball application will encompass. The featured infinity symbol represents the infinite possibilities that the 8Ball application will provide for Government of Canada employees, other government organizations, unregulated parties, and consumers. The ouroboros, an ancient symbol that depicts a serpent eating its own tail, symbolizes the positive feedback loop that the 8Ball application will harness. This feedback loop will allow the CPS program to gather and leverage more detailed information input by inspectors and consumers in order to compile further data for future surveillance and developmental purposes. Finally, the circular black and white 8Ball captures the foundation of the 8Ball project, inspectors will be able to “ask a question” and the application will provide an instantaneous response with advice.
User Interface
Initial sketches for the user interface included demonstrations of OCR capabilities and the organization of data entries. The images below outline the translation from primary sketches to current interactive functions of the developing 8Ball application. These images capture spectific functions of 8Ball, insight into the application's process flow and further capabilities is available in the 8Ball teaser video.
Benefits to Health Canada
The real-time compliance verification of products that the 8Ball Project presents will allow for numerous gains in accuracy, efficiency, and outreach such as:
- Ability to generate accessible reports that inspectors can share with regulated parties to promote outreach and regulation compliance
- Generated data from inputs allow for further and more thorough data analysis for future surveillance
- Provide a cohesive platform for inspectors to quickly and remotely access information
- Allows CPS to use the photographs taken by users to store the most up to date information on products such as accurate lot /model numbers, bar codes, product descriptions etc.
- Increase consumer confidence and government transparency by making information more easily accessible to members of the unregulated parties
Soft Launch
The soft launch comprised six inspectors from Quebec, Ontario, and British Columbia regions to use the application to scan cosmetic products in their duties as inspectors and specifically if they went onsite to international mail centres from December 2022 to February 2023.
Overall Feedback
The soft launch ended with approximately 63 scans of cosmetic products between the six participants.
The six participants conducted a follow-up survey where they indicated that the 8ball APP “made their job easier”
Specific comments include:
- The thought of being able to use an app to help identify hotlist ingredients for cosmetic products without having to go to the Hotlist website or shared drives and search for it saves a lot of time”, and
- The first impression of Soft Ball was very good as it is a useful tool which can make work easier for Inspectors”.
Speed Concern
During testing, there were also concerns and comments raised about the speed of the application. Five of the six participants reported bugs were related to the speed or completion of a submitted scan. This was brought up with the developers after the soft launch was completed to correct the bug.
In response to the bugs reported regarding the APP not being able to provide submission results in a timely manner, testers from ON, PR, BC and NCR regions tested on both mobile and desktop version of the 8ball APP with developers on March 10, 2023. Test results demonstrated the application was unable to retrieve submission within a reasonable time, causing back-end resubmissions and eventual crashes. During the test, it was proven the internet and cellular data speed were not the main factors causing the delays in processing speed of the APP. This indicator provides a level of confidence that the application can be supported in warehouses and mail centres where internet connectivity may be reduced.
The APP was updated on March 21, 2023, after a planned meeting with the developers and the soft launch team. After the update, the APP showed significant speed improvements in the cloud side “Submission to assessed” analysis which can help provide more use of the APP during onsite inspections. Further testing is required to help determine the extent of the update.
Time Saving
To determine the time saved using the APP compared to manually typing and searching for ingredients, 20 submissions were selected from the 63 scans during the soft launch for analysis. Students from Ontario region were asked to randomly select 20 submissions and compare their typing speed to the APP’s processing speed. Figure 1 illustrates the average processing time for the APP was 200.11 seconds (3.3 minutes) while the manual process time was on average 262.96 seconds (4.4 minutes). Outliers where the APP was taking an exceptionally long time to process is highlighted in red font in Figure 1. One potential reason for the outliers is the spin up time for the cloud server for the application. Each outlier was the first scan of each batch the tester completed. Further testing can show if this hypothesis is correct, or another issue is the cause for the outlier. The bolded fonts under the ‘Time Difference (APP-Manual in sec)’ column reflect the submission where the APP processing time was longer than manual processing.