2
edits
Changes
mLine 1:
Line 1: − + + + + + + + + + + + + + + + − Project Cipher is a [https://www.gcpedia.gc.ca/wiki/Health_Canada_Innovation_and_Experimentation Solutions Fund] enabled experiment that was divided into two streams. + + + − + − + − The project team was successful in developing and building an AI tool that can perform the following functions: − * Risk rate inspection observations + − * Link inspection observations to a regulatory section Assign a standard line to inspection observations − * Provide a machine generated overall compliance score to [https://www.canada.ca/en/health-canada/services/drugs-health-products/natural-non-prescription/legislation-guidelines/guidance-documents/good-manufacturing-practices.html GMP] inspection reports. − Cipher has the ability to change the way we currently do business. The tasks executed by Cipher are currently performed by inspectors and the outputs of these tasks could vary depending on factors such as industry experience, inspection experience, training, and geographical location of the inspector. + − Some of the key benefits of a tool like Cipher include: + + Line 27:
Line 41: − + − Applying the exploratory design experimentation model and working in the experimentation space allowed the project team to quickly transform an idea into a fully developed AI tool. This space provided an environment where the team could develop the application in a flexible, agile environment. This approach significantly contributed to project success. Working in this space allowed the project team and the developers to build, test, and improve the Cipher application in a timely manner and resulted in achieving significant accomplishments in a short period of time. + + + + + + + − We recognized how invaluable it was to build partnerships with Industry and other Departments to leverage the necessary expertise in AI and Machine Learning technologies and to broadcast project success outside of [[ROEB-Digital Transformation|ROEB]]. Utilizing short development cycles and early testing, we were able to make modifications to the application and identify areas of improvements for future iterations. + − By creating a collaborative experimentation space, we were able to build support for the project and the AI tool by engaging users and collecting their feedback throughout the project lifecycle. Engaging users early in the development phase ensured that the user’s business requirements were met. This approach also allowed the project team to build trust with users, which will help garner support to future operationalize the Cipher tool. − There is a sense of positivity surrounding Cipher and users are truly excited about the project and the tool! Inspectors and other potential users are saying “Cipher is the first tool that looks like it belongs in the 21st Century” and “Cipher is the Future”. +
The page was updated to reflect the current status of the project. All sections were revised accordingly.
[[Health Canada]] conducts on-site inspections of manufacturing, packaging, labeling, importing, wholesaling and distributing establishments for numerous health product lines: drugs, medical device, biologics and cannabis. These inspections produce written reports that details observations, non-compliance issues and regulatory decisions specific to individual establishments. Although this process satisfies the inspection activity within our regulatory framework, there is an opportunity to leverage the power of analysis with continuous data to inform horizontal programs with greater insight. Regulatory Operations and Enforcement Branch initiated Project Cipher to take an innovative approach to better understand and leverage historical inspection data to simplify the day-to-day tasks of inspectors and make their jobs easier.
[[Health Canada]] is responsible for the administration and enforcement of the ''Food & Drug Regulations,'' which includes conducting Good Manufacturing Inspections ([https://www.canada.ca/en/health-canada/services/drugs-health-products/natural-non-prescription/legislation-guidelines/guidance-documents/good-manufacturing-practices.html GMP]) of licensed establishments including manufacturers, packagers, labelers, importers, wholesalers, importers and distributors. GMP inspectors are responsible for conducting on-site inspections and reviewing inspection reports from other regulators. Both activities involve writing inspection observations/reports, and deciding the overall compliance status for drug establishments. Although this process satisfies the inspection activity within our regulatory framework, there is an opportunity to leverage the power of analysis with continuous data to inform horizontal programs with greater insight.
The Transformation Office (TO) under the Planning and Operations Directorate (POD) in partnership with Health Product Inspection Licensing (HPIL) initiated Project Cipher to take an innovative approach to better understand and leverage historical inspection data to simplify the day-to-day tasks of inspectors and make their jobs easier.
Cipher is a new machine learning tool that can assist inspectors perform the following functions:
1) Risk rate observations
2) Link each observation to a regulatory section
3) Assign a standard line to inspection observations
4) Provide a machine generated overall compliance score to GMP inspection reports.
This will allow the program(s) to better leverage their data, assist inspectors with their risk-based decisions, and reduce the amount of time to write inspection observations and reports.
=== EXPERIMENTATION ===
=== EXPERIMENTATION ===
Cipher was created in the [https://www.gcpedia.gc.ca/wiki/Health_Canada_Innovation_and_Experimentation Solutions Fund] Experimentation Design space by ROEB. The solutions fund design space provided an environment where the team could develop the tool in a flexible, agile environment. This approach significantly contributed to project success where an idea was quickly transformed into a fully developed machine learning tool. Leveraging the Solutions Fund design space, developers utilized short development cycles and early testing to quickly build, test, and improve the tool.
Cipher development and experimentation was divided into two streams.
* Stream I of Project Cipher (Explore the Problem) proposed to conduct an in-depth analysis to examine the value of applying [[wikipedia:Artificial_intelligence|Artificial Intelligence]] to the corpus of data, in an attempt to leverage the results to support risk based (and predictive) decision making.
* Stream I of Project Cipher (Explore the Problem) proposed to conduct an in-depth analysis to examine the value of applying [[wikipedia:Artificial_intelligence|Artificial Intelligence]] to the corpus of data, in an attempt to leverage the results to support risk based (and predictive) decision making.
* Stream II (Experimentation/Testing) was focused on improving inspection processes by leveraging artificial intelligence and machine learning. This includes building and developing a machine learning tool for compliance and enforcement staff to assist them making risk based decisions and compliment the work these people do on a daily basis.
* Stream II (Experimentation/Testing) was focused on improving inspection processes by leveraging artificial intelligence and machine learning. This includes building and developing the current machine learning tool to assist compliance and enforcement staff with their risk based decisions and compliment their day-to-day work.
Collaboration was critical during experimentation. By creating a collaborative experimentation space user engagement and feedback was critical throughout the project development lifecycle. Engaging users early in the development phase ensured that the user’s business requirements were met and built trust amongst users.
The next step is to operationalize the Cipher tool across Health Canada programs, starting with the Health Product Inspection Licensing (HPIL) unit.
=== BENEFITS TO HEALTH CANADA ===
=== BENEFITS TO HEALTH CANADA ===
'''''“Cipher is the Future”''''' & '''''“Cipher is the first tool that looks like it belongs in the 21<sup>st</sup> Century”''''' are examples of feedback received by inspectors and other potential users.
Cipher has the ability to change the way we currently do business. The tasks executed by Cipher are currently performed by inspectors and the outputs of these tasks could vary depending on factors such as industry experience, inspection experience, training, and geographical location of the inspector.
Some of the key benefits of a tool like Cipher include:
* Allows Health Canada to better leverage historical data.
* Allows Health Canada to better leverage historical data.
* Can improve operational processes
* Can improve operational processes
* Can potentially reduce the need for some Health Canada guidance documents and/or standard operating procedures.
* Can potentially reduce the need for some Health Canada guidance documents and/or standard operating procedures.
=== OUR EXPERIENCE ===
=== NEXT STEPS ===
Project Cipher has completed initial testing on the minimum viable product (MVP) by Inspectors from the Health Product Inspection & Licensing Unit. Improvements to the tool were made based on valuable feedback from inspectors. With the completion of Stream 2 in the Solutions Fund, we have support from the branch executive committee (BEC) to operationalize Cipher as a pilot project. Before the tool can be piloted the following key improvements are required to bring the product into production:
'''Step 1: IT Infrastructure/Protected B cloud environment'''
The number 1 priority for project Cipher is to secure IT infrastructure and a protected B cloud hosting for the current tool which is built using Python, Fast Text, Linux, and Docker. This is required as soon as possible to complete testing of the tool and deploy the it within ROEB programs. All major milestones and deliverables for the project are highly dependent on successfully securing a protected B cloud environment.
'''Step 2: Develop and deploy a new iteration of Cipher'''
POD-TO and HPIL plan to partner to develop and build a new version of the Cipher tool for future use cases (I.e. foreign evidence review use case). The technology for future iterations of Cipher have not been determined and will depend on future testing and experimentation using the current tool.
'''''“Cipher is the future”''''' and is capable of transforming operational processes. Cipher is truly an innovative, transformative project that is aligned with ROEB’s Build Back Better (BBB) campaign. Cipher is just scratching the surface on how AI/machine learning technologies can transform operational processes and the tool could have many other benefits. The biggest benefit of a tool like this is that it is easily scalable and can be leveraged by other ROEB programs, across regulatory departments within Health Canada or the Government of Canada. Long term it could even be potentially used by other global regulatory partners.
== Repository ==
== Repository ==
Click [https://022gc.sharepoint.com/:f:/s/ROEB-POD-TransformationOffice/ErwX1kWV2T5ClFdcvesiQhEBKFa306Do5-qiSuJqByN5bA?e=sjAJCF here] to access the Cipher files.
Click [https://022gc.sharepoint.com/:f:/s/ROEB-POD-TransformationOffice/ErwX1kWV2T5ClFdcvesiQhEBKFa306Do5-qiSuJqByN5bA?e=sjAJCF here] to access the Cipher files.