Regulatory Operations and Enforcement Branch, Technology and Business Innovation (TBI) unit, in partnership with Health Product Inspection Licensing (HPIL) initiated Project Cipher to take an innovative approach to better understand and leverage historical inspection data to simplify the day-to-day tasks of inspectors and make their jobs easier.
Health Canada conducts on-site inspections of manufacturing, packaging, labeling, importing, wholesaling and distributing establishments for numerous health product lines: drugs, medical device, biologics and cannabis. These inspections produce written reports that details observations, non-compliance issues and regulatory decisions specific to individual establishments. Although this process satisfies the inspection activity within our regulatory framework, there is a lost opportunity to leverage the power of analysis on continuous data to inform horizontal programs with greater insight.
EXPERIMENTATION
Project Cipher is a Solutions Fund enabled experiment that was divided into two streams.
- Stream I of Project Cipher (Explore the Problem) proposed to conduct an in-depth analysis to examine the value of applying Artificial Intelligence to the corpus of data, in an attempt to leverage the results to support risk based (and predictive) decision making.
- Stream II (Experimentation/Testing) was focused on improving inspection processes by leveraging artificial intelligence and machine learning. This includes building and developing a machine learning tool for compliance and enforcement staff to assist them making risk based decisions and compliment the work these people do on a daily basis.
The project team was successful in developing and building an AI tool that can perform the following functions:
- Risk rate inspection observations
- Link inspection observations to a regulatory section Assign a standard line to inspection observations
- Provide a machine generated overall compliance score to GMP inspection reports.
BENEFITS TO HEALTH CANADA
Cipher has the ability to change the way we currently do business. The tasks executed by Cipher are currently performed by inspectors and the outputs of these tasks could vary depending on factors such as industry experience, inspection experience, training, and geographical location of the inspector.
Some of the key benefits of a tool like Cipher include:
- Allows Health Canada to better leverage historical data.
- Can improve operational processes
- Can assist inspectors with their decision making
- Can be used as a training tool
- Cipher can generate comparable predictions/results to inspectors that can be improved over time.
- Can improve consistency throughout the program and therefore improving rapport with stakeholders
- Can potentially reduce the amount of time required prepare inspection reports, therefore freeing up inspectors for higher risk activities.
- Can potentially reduce the need for some Health Canada guidance documents and/or standard operating procedures.
OUR EXPERIENCE
Applying the exploratory design experimentation model and working in the experimentation space allowed the project team to quickly transform an idea into a fully developed AI tool. This space provided an environment where the team could develop the application in a flexible, agile environment. This approach significantly contributed to project success. Working in this space allowed the project team and the developers to build, test, and improve the Cipher application in a timely manner and resulted in achieving significant accomplishments in a short period of time.
We recognized how invaluable it was to build partnerships with Industry and other Departments to leverage the necessary expertise in AI and Machine Learning technologies and to broadcast project success outside of ROEB. Utilizing short development cycles and early testing, we were able to make modifications to the application and identify areas of improvements for future iterations.
By creating a collaborative experimentation space, we were able to build support for the project and the AI tool by engaging users and collecting their feedback throughout the project lifecycle. Engaging users early in the development phase ensured that the user’s business requirements were met. This approach also allowed the project team to build trust with users, which will help garner support to future operationalize the Cipher tool.
There is a sense of positivity surrounding Cipher and users are truly excited about the project and the tool! Inspectors and other potential users are saying “Cipher is the first tool that looks like it belongs in the 21st Century” and “Cipher is the Future”.
Repository
Click here to access the Cipher files.