Project Cipher
Health Canada is responsible for the administration and enforcement of the Food & Drug Regulations, which includes conducting Good Manufacturing Inspections (GMP) of licensed establishments including manufacturers, packagers, labelers, importers, wholesalers, importers and distributors. GMP inspectors are responsible for conducting on-site inspections and reviewing inspection reports from other regulators. Both activities involve writing inspection observations/reports, and deciding the overall compliance status for drug establishments. Although this process satisfies the inspection activity within our regulatory framework, there is an opportunity to leverage the power of analysis with continuous data to inform horizontal programs with greater insight.
The Transformation Office (TO) under the Planning and Operations Directorate (POD) in partnership with Health Product Inspection Licensing (HPIL) initiated Project Cipher to take an innovative approach to better understand and leverage historical inspection data to simplify the day-to-day tasks of inspectors and make their jobs easier.
Cipher is a new machine learning tool that can assist inspectors perform the following functions:
1) Risk rate observations
2) Link each observation to a regulatory section
3) Assign a standard line to inspection observations
4) Provide a machine generated overall compliance score to GMP inspection reports.
This will allow the program(s) to better leverage their data, assist inspectors with their risk-based decisions, and reduce the amount of time to write inspection observations and reports.
EXPERIMENTATION
Cipher was created in the Solutions Fund Experimentation Design space by ROEB. The solutions fund design space provided an environment where the team could develop the tool in a flexible, agile environment. This approach significantly contributed to project success where an idea was quickly transformed into a fully developed machine learning tool. Leveraging the Solutions Fund design space, developers utilized short development cycles and early testing to quickly build, test, and improve the tool.
Cipher development and experimentation was divided into two streams.
- Stream I of Project Cipher (Explore the Problem) proposed to conduct an in-depth analysis to examine the value of applying Artificial Intelligence to the corpus of data, in an attempt to leverage the results to support risk based (and predictive) decision making.
- Stream II (Experimentation/Testing) was focused on improving inspection processes by leveraging artificial intelligence and machine learning. This includes building and developing the current machine learning tool to assist compliance and enforcement staff with their risk based decisions and compliment their day-to-day work.
Collaboration was critical during experimentation. By creating a collaborative experimentation space user engagement and feedback was critical throughout the project development lifecycle. Engaging users early in the development phase ensured that the user’s business requirements were met and built trust amongst users.
The next step is to operationalize the Cipher tool across Health Canada programs, starting with the Health Product Inspection Licensing (HPIL) unit.
BENEFITS TO HEALTH CANADA
“Cipher is the Future” & “Cipher is the first tool that looks like it belongs in the 21st Century” are examples of feedback received by inspectors and other potential users.
Cipher has the ability to change the way we currently do business. The tasks executed by Cipher are currently performed by inspectors and the outputs of these tasks could vary depending on factors such as industry experience, inspection experience, training, and geographical location of the inspector.
Some of the key benefits of a tool like Cipher include:
- Allows Health Canada to better leverage historical data.
- Can improve operational processes
- Can assist inspectors with their decision making
- Can be used as a training tool
- Cipher can generate comparable predictions/results to inspectors that can be improved over time.
- Can improve consistency throughout the program and therefore improving rapport with stakeholders
- Can potentially reduce the amount of time required prepare inspection reports, therefore freeing up inspectors for higher risk activities.
- Can potentially reduce the need for some Health Canada guidance documents and/or standard operating procedures.
NEXT STEPS
Project Cipher has completed initial testing on the minimum viable product (MVP) by Inspectors from the Health Product Inspection & Licensing Unit. Improvements to the tool were made based on valuable feedback from inspectors. With the completion of Stream 2 in the Solutions Fund, we have support from the branch executive committee (BEC) to operationalize Cipher as a pilot project. Before the tool can be piloted the following key improvements are required to bring the product into production:
Step 1: IT Infrastructure/Protected B cloud environment
The number 1 priority for project Cipher is to secure IT infrastructure and a protected B cloud hosting for the current tool which is built using Python, Fast Text, Linux, and Docker. This is required as soon as possible to complete testing of the tool and deploy the it within ROEB programs. All major milestones and deliverables for the project are highly dependent on successfully securing a protected B cloud environment.
Step 2: Develop and deploy a new iteration of Cipher
POD-TO and HPIL plan to partner to develop and build a new version of the Cipher tool for future use cases (I.e. foreign evidence review use case). The technology for future iterations of Cipher have not been determined and will depend on future testing and experimentation using the current tool.
“Cipher is the future” and is capable of transforming operational processes. Cipher is truly an innovative, transformative project that is aligned with ROEB’s Build Back Better (BBB) campaign. Cipher is just scratching the surface on how AI/machine learning technologies can transform operational processes and the tool could have many other benefits. The biggest benefit of a tool like this is that it is easily scalable and can be leveraged by other ROEB programs, across regulatory departments within Health Canada or the Government of Canada. Long term it could even be potentially used by other global regulatory partners.
Repository
Click here to access the Cipher files.