What is Regulatory Experimentation?

Centre for Regulatory Innovation: What is Regulatory Experimentation?

Home	Events	What is Regulatory Experimentation?	Regulatory Experimentation Expense Fund	Regulators' Capacity Fund	Completed CRI Supported Projects	Tools and Resources

Overview

Experimentation can mean different things in different contexts to different people. There are various approaches to what constitutes an experiment among researchers, innovators and evaluation practitioners. At its loosest, an ‘experiment’ is sometimes used to mean ‘trying out something new’. Strict definitions, on the other hand, consider a process an experiment only if it generates evidence that meets very specific criteria. The Centre for Regulatory Innovation’s (CRI) focus is on regulatory experimentation. Our definition is:

A regulatory experiment is a test or trial of a new product, service, approach or process designed to generate evidence or information that can inform the design or administration of a regulatory regime.

In practice, a regulator might experiment with:

• A regulated product or service. This includes new products, services and business models that create regulatory uncertainty. These may be unregulated or prohibited in some way by existing regulation. The experiment might be designed to understand the implications of those innovations if they were used in the real world or assess how easy they would be to regulate using existing mechanisms.
• A new approach to regulating. In some cases, regulators may want to trial a new version of a regulation under controlled conditions in order to monitor its effects. For example, regulators could test proposed regulations with a specific group of regulated entities to assess their effectiveness before being formally implemented.
• A regulatory process. This covers a broad range of activities. For example, a regulator could test different ways of consulting stakeholders on the design of new regulations, such as co-development, or it could run an experiment to understand if a new process for compliance or enforcement is better than the current procedures in place.

What are the benefits of regulatory experimentation?

The core reason to undertake regulatory experimentation is to provide evidence or information to inform regulatory decision making. Experimentation can complement other approaches such as post-implementation evaluation, or research and theory to make decisions. Experimentation and experiments can be useful to regulators for several reasons:

1. They provide a structured approach to systematically generate high quality evidence to inform regulators’ decisions. Experimentation provides a way in which valuable new information (e.g. how well a new regulatory approach works) can be generated through a structured process. Building a robust evidence base is an essential component of better regulation.
2. They are a powerful way to test alternative approaches and to identify potentially better approaches to achieving regulatory objectives. This might include new regulatory approaches (e.g. for a new technology, product or service) or improvements to existing regulatory approaches (e.g. to reduce the compliance burden without increasing compliance risk).
3. Experiments help regulators to reduce the risk and uncertainty associated with a new regulatory approach prior to implementing it at scale. An experiment may be the only chance to observe, control and learn before a regulatory approach is fully implemented and may lead to irreversible consequences.
4. Experiments allow policy makers and regulators to generate information in a controlled and purposeful way. Experiments allow for a much greater degree of control and more effective monitoring to isolate aspects of particular interest than is possible in a ‘real world’ setting (where there is greater complexity).
5. Experiments can generate information in complex, changing systems. They can be particularly useful in rapidly changing circumstances where previous research and theory may become quickly outdated or offer little guidance.
6. By building a more robust evidence base they help to build consensus around a particular regulatory option or issue that may otherwise be contested.

Regulatory Sandboxes

Rather than initiating a stand-alone experiment itself, regulators may want to facilitate experimentation and information-gathering activities in cooperation with third parties. Innovative products or services may pose challenges for regulators as it may be unclear how regulations apply or these products may present a radical challenge to the regulatory framework. In these situations Innovators and regulators will have an interest in generating evidence and information about how an innovation will work in a ‘real world’ context (e.g. with real users, or in a realistic physical environment). Setting up a regulatory sandbox is one way regulators can facilitate these experimentation and information gathering activities. We define a regulatory sandbox as:

A facility, created and controlled by a regulator, designed to allow the conduct of testing or experiments with novel products or processes prior to their full entry into the marketplace.

The CRI and regulatory experimentation

The CRI’s goals include creating opportunities for regulatory experimentation and helping federal regulators embed experimental learnings into regulation. The CRI can support regulators with the design and implementation of regulatory experiments through guidance and funding available through the Regulatory Experimentation Expense Fund. For further information please contact the CRI at cri-cir@tbs-sct.gc.ca.