Assessing, Selecting, and Implementing Instruments for Government Action

Introduction

A broad spectrum of instruments for government action exists for advancing public policy. These instruments range from laws to economic, public, and peer pressure. Choosing an instrument or, more importantly, the appropriate mix of instruments involves selecting the tools that are most likely to achieve the public policy objective pursued on a sustained basis and at an acceptable cost.

Several federal documents provide direction and guidance on selecting instruments. These include the Canadian Cost-Benefit Analysis Guide, the Cabinet Directive on Law-Making, the Guide to Making Federal Acts and Regulations, and the Cabinet Directive on Streamlining Regulation.

In September 2004, the External Advisory Committee on Smart Regulation (EACSR) tabled its report, which included a recommendation that a framework be designed for assessing and selecting instruments, including economic instruments, for government action. With the coming into force of the Cabinet Directive on Streamlining Regulation in April 2007, departments and agencies are now required to select an appropriate mix of government instruments.

This document consolidates the government's various guidelines and directives on instruments for government action and addresses the EACSR's recommendation by providing a decision-making framework for assessing, selecting, and implementing instruments.

Purpose

This document is intended to support the Cabinet Directive on Streamlining Regulation. It provides guidance to officers of the Treasury Board of Canada Secretariat who perform a challenge function regarding policy, legislative, and regulatory proposals involving issues of instrument choice and to officials in other government departments and their legal advisors whose work involves considering which instruments will best achieve policy objectives.

Document Organization

This document has two sections:

• Section 1 provides background and context for the question of instrument choice. It identifies an array of instruments and focusses on how instrument choice can lead to policy innovation.
• Section 2 presents an analytical framework to facilitate a disciplined approach to assessing, selecting, and implementing instruments. The framework sketches out a sequence of enquiry, suggests a methodological foundation, and provides guidance for each step in the instrument choice process. Departments and agencies are invited to use the framework as is or as a template for developing their own versions in their respective areas of responsibility.

Annex A provides an example of one department's instrument choice framework: Environment Canada's Qualitative Screening of Management Tools (QSMT). Eventually, this annex will be expanded to include a series of weblinks to departments' customized instrument choice frameworks. For the time being, Annex B presents detailed tables for Environment Canada's QSMT, for illustrative purposes.

Key Messages

• Instrument choice should be considered early in the policy development process.
• The government cannot deal with every situation. Its involvement must be assessed in light of its responsibilities, its resources, and the likely effectiveness of its involvement relative to that of a variety of actors such as other governments, the private sector, non-governmental organizations, and the voluntary sector.
• A broad range of instruments exists, allowing the government to choose the type and degree of intervention, if any.
• A mix of instruments has been found to be effective in achieving successful outcomes.
• The effectiveness of an instrument in promoting conforming behaviour needs to be considered early in the policy development process.
• A statute or regulation should be chosen only after the full range of possible instruments has been considered.

Section 1: Background and Context

What are instruments for government action?

Instruments for government action are the means a government has at its disposal to achieve public policy outcomes—to govern. While several definitions of "instruments for government action" exist, this document uses a broad interpretation, defining them as the "means by which policy objectives are pursued." Instruments for government action set up relationships between the state and its citizens. In some cases, such as criminal law, the relationship is of a coercive nature. In other cases, such as legal agreements, the relationship is reciprocal.

Some well-known instruments that can be used singly or in combination are:

• laws (statutes and regulations)
• economic instruments, including market-based instruments, taxes, fees, user charges, loans and loan guarantees, and public expenditure
• public ownership
• forms of self-regulation, including regimes operating within a framework of legislative authority
• standards and other forms of voluntary action
• performance-based regulation
• contracts
• information and education
• insurance schemes
• collaborative/consensual approaches, including formalized partnerships and less formalized networks

"Regulation refers to the diverse set of instruments by which governments set requirements on enterprises and citizens. Regulation includes laws, formal and informal order and subordinate rules issued by all levels of government, and rules issued by non-governmental bodies or self-regulatory bodies to whom governments have delegated regulatory powers."

-OECD report on Canada: Maintaining Leadership Through Innovation, 2002.

Regulation is one of the most important instruments used by governments. It may be defined as government intervention through a set of rules identifying permissible and impermissible activity on the part of individuals, firms, or government departments and agencies, along with accompanying sanctions and rewards. This definition, based on one from The Tools of Government Workbook 1, Social Regulation Workbook (in The Tools of Government Workbooks, ed. Lester M. Salamon, Oxford University Press (2002)), emphasizes the use of legal texts known as acts (primary legislation) and regulations (secondary legislation). However, too often government intervention is equated with legislation, which means that not enough consideration is given to other forms of government action. A broader definition of regulation is needed to understand the breadth and complexity of instruments required to make any regulatory regime function effectively.

The analogy between tools in a tool box and instruments in a government's policy tool kit demonstrates the point. If all you have is a hammer to work with, you will look for things to hammer and use the hammer for everything you find. If legislation is the only tool a government uses, it will deal with every public issue by passing yet another law.

Many governments are now considering instruments other than regulation to achieve public policy outcomes. Prompted by factors such as globalization, international competitiveness, increased emphasis on market solutions, and new philosophies of governance, they are seeking new or modified instruments that provide effective approaches to policy making. Some of the new policy instruments or new applications of policy instruments that government is now using and that are indicative of the government's recent emphasis on the search for new governing tools are:

• adjustment programs
• capacity building
• commercialization
• contracting out
• decentralization
• partnerships
• networks
• organizational structure

A focus on results

Government officials are encouraged to adopt a more comprehensive approach to developing proposals for attaining policy objectives. They should focus on achieving a desired outcome rather than assuming that a particular instrument, particularly an act or regulation, will be effective. The Cabinet Directive on Law-Making states that:

Law [statutes and regulations] should be used only when it is the most appropriate [instrument]. When a legislative proposal is made to the Cabinet, it is up to the sponsoring Minister to show that this principle has been met, and there are no other ways to achieve the policy objectives effectively.

If the decision is made to use a law, consideration should be given to how the law is designed. The law should be conceived in ways that allow for the use of a range of approaches and compliance measures. Research shows that using a mix of instruments often improves outcomes. A package of instruments can be used with significant success to improve citizen quality of life, national competitiveness, and environmental protection.

In the regulatory area, concerns about the negative effects of conventional laws on industry innovation and competitiveness have made governments look to other forms of laws, such as performance-based regulation (PBR). PBR sets the standard or objective to be met rather than prescribing the means for achieving it. It also harnesses the energy of external actors in finding solutions. In 1996, the Quebec government stated by decree that it would favour the use of regulation by objectives (also known as performance-based regulation), preferring regulations that focus on the results sought by the legislature or regulatory authority to a precise definition of the means for achieving those results.

The ways in which instruments interact should also be considered when assessing and selecting instruments. Officials should explore how different instruments can most effectively be used together to support each other's strengths and weaknesses and avoid conflicts. Some inherently complementary combinations include:

• information and all other instruments
• voluntarism and command-and-control regulation
• command-and-control regulation and supply-side incentives
• command-and-control regulation and broad-based economic incentives
• contracting out
• decentralization
• partnerships
• networks
• organizational structure

Limits of Law

The following examples illustrate that laws alone are not always the best option.

Laws have been in place since shortly after World War I to dissuade people from drinking and driving, but no significant drop in the incidence of impaired driving occurred until the 1980s. Since prohibitive legislation seems to have been ineffective, what happened in the 1980s to alter behaviour?

One answer could be that government began to look at the problem differently. Through public education campaigns, federal and provincial intervention, and community and non-governmental organization involvement, a change in awareness and behaviour began to occur, causing a major shift in societal norms and attitudes, to the point where it is now socially unacceptable to drive while impaired.

In this example, government use of laws to forbid a type of behaviour was less successful than using a range of instruments that encourage individuals to think about their behaviour and the risks to others and to themselves, and to make different choices. Similarly, a case study on reducing the use of tobacco supports the view that using a mix of instruments leads to more successful outcomes.

Another example involves efforts to prohibit the use of cell phones while driving. A study conducted by the United States Insurance Institute for Highway Safety found that, in the months following the coming into force of legislation in New York, the percentage of drivers using cell phones dropped from 2.3 to 1.1 per cent. However, within 15 months and despite enforcement efforts, cell phone use among drivers climbed back up to 2.1 per cent. The Canada Safety Council is of the view that a similar law in effect in Newfoundland is practically unenforceable without substantially increasing the number of police officers, which, in an era of limited resources, is difficult to do.

Stimulating innovation through policy instruments

Careful consideration and appropriate use of non-conventional policy instruments may lead to innovative ideas and means for government intervention. Innovation is a process whereby new and different approaches to government intervention are incorporated into the decision-making framework and where, in applying this framework, knowledge sharing about the viability of different instruments in various circumstances acts as a catalyst for change in government action. Below are some situations where a non-conventional approach may be appropriate, with examples of where such an approach has proven successful.

1. To supplement conventional regulatory approaches, thus reducing the need for regulatory enforcement action through the courts

Example: The Competition Bureau, working with the Retail Council of Canada and three other major retail associations, developed a code pertaining to the accuracy of price scanners in supermarkets. Under the Scanner Price Accuracy Voluntary Code, if a customer finds a discrepancy between the price advertised and that charged at the counter, the customer is entitled to get the product for free if it is less than $10, or for $10 off if it is more than $10. This creates an incentive for both consumers and retailers to be vigilant about pricing. The code is an adjunct to regulatory approaches, as there are laws at both the federal and provincial levels prohibiting misleading or deceptive advertising. The code prompts retailers to comply with these laws and in effect mobilizes consumers to act as informal "inspectors" in regard to the practices of retailers. Evidence suggests that complaints about retailers to the Competition Bureau concerning inaccurate pricing have decreased since the introduction of the code.

2. To address problematic activity more quickly than through conventional approaches

Example: The Accelerated Reduction/Elimination of Toxics (ARET) program was introduced in 1994 as a challenge to industry to voluntarily reduce or eliminate the release of 117 identified toxic substances by the year 2000. For its time, ARET was an innovative approach, as the concept of voluntary measures was still uncharted territory in Canada. A primary objective was to stimulate early action on toxic substances without a full-blown risk assessment under the Canadian Environmental Protection Act. Eight major industry sectors, 171 companies and government organizations, and 318 facilities took the challenge. Collectively, they reduced the release of almost 28,000 tonnes of toxic substances, as measured by comparing releases in the last year of the program with the base-year levels of those releases. Over the entire course of the program, more than 70,000 tonnes of substances targeted by ARET were prevented from release. The use of an expanded tool box and non-conventional approaches means government can address problematic issues more quickly than through conventional regulations, which can be slow and costly to develop. Environment Canada and Health Canada are now in the process of reviewing hundreds of chemicals with a view to managing risk using a range of instrument options referred to in the Canadian Environmental Protection Act (see sidebar). Other substances that are not toxic but of concern are managed throughout their life cycle by means of a variety of non-regulatory instruments. Environment Canada now has a minister-endorsed policy for minimum criteria applied to all voluntary initiatives, to ensure that clear targets are achieved in an accountable way through verifiable agreements. These voluntary agreements will replace the ARET program.

Instruments Allowed Under the Canadian Environmental Protection Act

In addition to establishing regulation-making powers, the Canadian Environmental Protection Act provides authority for the use of economic instruments, environmental quality objectives, guidelines and codes of practice, administrative and equivalency agreements, pollution prevention plans, environmental emergency plans, information-gathering notices, national pollutant release inventory, alternative compliance measures known as environmental protection alternative measures, and environmental protection compliance orders.

3. To overcome or avoid federal-provincial jurisdiction issues

Example: In many policy contexts, both the federal and provincial governments have some degree of constitutional authority to develop legislation. This can lead to "paralysis," where neither level of government acts for fear of challenges that its legislation exceeds its jurisdictional competencies. Voluntary market-driven approaches are not so constrained. For example, an "ombudsnetwork" has been put in place by private-sector financial institutions to provide consumers of Canadian financial services with a complaint resolution service. Because the participating institutions may be federally or provincially regulated and because participation in the network is voluntary, the ombudsnetwork avoids any possible constitutional challenges. Federal and provincial regulators may still impose a regulatory approach at any time but, in the meantime, the private-sector approach serves the purpose.

4. To address fast-moving technologies or issues

Example: E-commerce is an area where technology is moving very quickly—perhaps too quickly for regulators to stay current through conventional regulatory approaches. To fill the gap, the private sector and consumer organizations have established voluntary market-driven schemes for merchant reliability, privacy, and redress. Under such schemes, a third party such as the Better Business Bureau or a trusted consumer organization allows businesses to use a "seal" or logo on their websites that tells consumers that the business has been checked for reliability and that, if something goes wrong with the purchase, they have a source of redress other than the legal system. This is particularly helpful in cases where a transaction crosses jurisdictions. Perhaps, at some point, an effective and cohesive inter-jurisdictional legal approach will be adopted but, in the meantime, voluntary approaches offer some assistance for consumers wishing to engage in cross-border e-transactions and for firms wishing to offer services across borders. In Canada, the Office of Consumer Affairs of Industry Canada is in the process of developing a voluntary code for consumer protection in e-commerce that could be used as the basis for seal programs.

5. To address issues that fall outside the legislative scope of government

Example: The Federal Contractors Program uses the government's procurement power to address employment equity in the workforce. As a condition for bidding on large federal contracts, contractors are required to certify in writing their commitment to employment equity. Entities that do not want to do business with the federal government are not required to comply with the program. As entering into business with the federal government is voluntary, this is an example of the government using its procurement power to further its policy aims and to stimulate changes in behaviour beyond its legislative competencies.

6. To allow governments to encourage activities that they wish to promote but not require

Example: The federal government's EcoLogo program and some aspects of its energy efficiency programs provide signals to consumers who wish to be more environmentally responsible, but do not impose legislative requirements on industry. Various cultural industry programs that provide financial incentives (including those for the film, magazine, book, and music industries) operate in this manner.

7. To provide Canadians with faster, more convenient, less intimidating redress options than going to court, thereby reducing the burden on courts

Example: The Canadian Motor Vehicle Arbitration Program (CAMVAP) is an industry-funded arbitration program for consumers who have problems with their cars. In the United States, "lemon laws" provide the same relief, but require the use of courts. In Canada, consumers may opt to use CAMVAP or the courts.

Section 2: Analytical Framework for Selecting Instruments

The analytical process for selecting instruments is defined by a number of steps all of which are supported by consultation. The first step in the process is the identification and definition of the problem. This is followed by the setting of objectives and then the identification of potential intervention points. Next, actors and institutions having an effect on risks and objectives are identified and instruments are selected and considered. Finally, performance indicators are set and then the instruments are implemented.

This section presents an analytical framework to facilitate a disciplined approach to assessing, selecting, and implementing instruments. Departments and agencies may use the framework as is or as a template for developing their own framework for their respective areas of responsibility.

The framework sketches out a sequence of enquiry, specifies a methodological foundation, and provides guidance for each step in the instrument choice process.

Benefits of using an analytical framework

Several benefits flow from the use of an analytical framework for selecting instruments:

• greater transparency in decision making by providing an explicit rationale for instrument choices
• greater cohesion in decision making by providing a disciplined approach for assessing and selecting instruments
• overcoming of risk aversion by using a risk-based analysis that will assist in understanding the challenges and the most appropriate means of addressing risks
• better outcomes by selecting an appropriate mix of instruments

Overarching rationales for the framework

The following framework is based on two overarching rationales:

• The process of analyzing a situation or problem and considering means by which the government could take appropriate action is iterative.
• The contribution of consultation throughout this iterative process is crucial. It enhances government transparency, promotes knowledge sharing, and supports the integrity of government action.

The framework is not intended to be a sequential roadmap of where and how officials should assess instruments to achieve public policy objectives. The process is inherently iterative in that the accumulation of information and knowledge concerning a problem or situation and the objectives that the government is aiming to achieve will require officials to repeatedly revisit each of the steps in the framework.

Given the complexity of policy making, ongoing consultation with actors and institutions that can have an effect on the risks and objectives being addressed is a crucial component of this iterative process. Consultation not only improves the government's instrument selection capacity, but also increases Canadians' trust in government and encourages knowledge sharing among government, industry, and non-governmental organizations.

Seeking an Optimal Policy Mix

The Conference Board of Canada considers stakeholder participation crucial to identifying issues and risks (problem identification), assessing and selecting instruments, and evaluating impacts. The Board proposes the following approach for environmental policy making:

• Develop objective: Consider the needs and objectives of all stakeholders [...]. Stakeholders include those contributing to the problem, those affected by the problem, and those with the potential to help address the problem.
• Select instruments: Consider all stakeholders, particularly in terms of their rights, roles, and responsibilities.
• Evaluate impacts: Consider all stakeholders who may be positively or negatively affected.

-Conference Board of Canada, The Optimal Policy Mix: Matching Ends and Means in Environmental Policy Making, 2000.

Step 1: Identify and define the problem: issues, risks, causal factors

The first step in any policy analysis is to identify and define the key features and sources of the problem. The knowledge generated at this stage is crucial to making good decisions concerning policy priorities, instrument choice, performance expectations, and (if "rules" are part of the instrument mix) strategies to achieve conforming behaviour.

Increasingly, the problems that governments are asked to deal with are characterized in terms of "risk." Governments have to make decisions about whether and how to intervene in situations involving a high degree of uncertainty regarding what is going on now and what may happen in the future. Risk, which can have both positive and negative connotations, is a concept that can help decision makers cope with the uncertainty inherent in governance activities, discipline public policy analysis, and foster more rational approaches to instrument choice.

Government intervention in response to a problem cannot change the past or even the present; it can only change the future. Moreover, government's ability to manage risks in the near term is often quite limited in that the die is already cast and a government's ability to devise and activate an appropriate mix of instruments is subject to significant time delays. The further a government's action extends into the future, the greater is the government's ability to influence the level of risk, but also the greater is the uncertainty about how the underlying systems that determine the risk will evolve.

To appropriately address a public policy issue, government needs a sense of where the problem is headed, how far and how fast–the dynamics of risk. Basing decisions on "snapshot" assessments of past or current risk makes sense if the risk is static and likely to remain that way throughout the time horizon relevant for public policy choices and program implementation. However, the risks that government must address are usually the result of complex interactions of social, economic, and environmental systems. These systems themselves are in a constant state of flux, which means that the risk could change over time and perhaps in ways that are not intuitively obvious.

Defining the Problem

How a problem is defined influences the selection of instrument(s) to address that problem. A simple example would be that of panhandling in the streets. If panhandling is defined as a nuisance and a potential danger, one option is to use law to prohibit the activity. However, if panhandling is seen as part of a larger problem such as poverty, lack of education, disability, etc., the intervention instruments will be different and may include outreach programs, workplace training, and/or other types of community support programs.

Playground Injuries

In most circumstances, risks are not constant and are likely to change over time, both in an absolute sense and relative to each other. This should be kept in mind during the problem analysis stage. Examining how the risk evolved in the past can provide important insights into how it may behave in the future, but investigating why it has changed may be even more important. Because correlations or linear projections of past trends do not always predict the future path, it may be prudent to make the effort to understand the causal influences that determine risk and how they function together as a system. Understanding why risk has changed in the past is an essential foundation for informed policy priority setting and for selection of instruments that will make a difference in how the risk evolves in the future. Developing knowledge regarding the systemic determinants of risk is the first step in any rigorous risk analysis and, in one form or another, has always been the hallmark of exemplary policy analysis.

In keeping with the Cabinet Directive on Streamlining Regulation and the Canadian Cost-Benefit Analysis Guide, a dynamic risk analysis methodology could be an integral step in the development of a rational approach to instrument choice. In reality, dynamic risk analysis is just good, disciplined public policy analysis, recast in "integrated risk management" terms.

Dynamic risk analysis involves describing how the risks have evolved over time, why the risks have occurred, the current level of risk, and how the risks are likely to evolve in the future with and without government intervention. What is most important is understanding the way in which the risks are evolving and the reasons for that evolution. In most cases, the creation of a logic model can help comprehension of how the causal determinants of risk affect ultimate outcomes.

Dynamic risk analysis also explicitly and transparently acknowledges the uncertainty inherent in public policy analysis, including limitations on the government's ability to accurately assess risk levels over time, understand the complex interactions of causal influences, and predict how intervention may change the trajectory of risk in the future.

The dynamic risk analysis approach provides a structured, highly disciplined, more realistic basis for performance assessments of intervention, with a focus on continuous learning and improvement in both policy design and program implementation.

Step 2: Set objective(s)

The second step involves articulating the policy goal(s) and desired outcomes. There may be several policy goals, each with a variety of outcomes. While policy goals and outcomes go hand in hand, they are not quite the same. For example, a goal might be to make a particular activity safer, while the desired outcome might be a 30-per-cent reduction in the rate of injury.

Goal versus Outcome

Setting goals and outcomes involves addressing the question of which risks are worth reducing and to what levels, while trying to balance the risks, costs, and benefits of action. Further guidance on balancing risks, costs, and benefits can be found in the Canadian Cost-Benefit Analysis Guide. This process will help to identify the instruments that will best manage the risks and will ensure that policy considerations drive the choice of instrument (ends drive the means), rather than the other way around.

A policy goal articulates desired risk management outcomes that would be acceptable in the public interest. The benefit incurred through intervention can be represented by the gap between the projected evolution of risk (i.e. in the absence of intervention) and the desired risk management performance that will be achieved over time as a consequence of the policy intervention. Performance indicators (discussed in Step VI) provide an indication of the effectiveness of an instrument or mix of instruments in achieving the desired risk management policy goal on an ongoing basis.

One avenue that should always be considered is that of taking no action. The idea that government needs to act assumes there must be a response to every situation. A rational approach to instrument choice includes the option of deliberately doing nothing. Maintaining the status quo or refraining from intervening may appear to be a non-decision, but when properly evaluated and applied in the right circumstances, it is a legitimate response known as a "static response."

As mentioned previously, adjustments may have to be made to the objectives as information and analysis evolve over the next steps in the process.

Step 3: Identify potential intervention points

When considering a problem and objectives, it may be possible to identify different points at which policy intervention could occur. Taking into account the actors and institutions that could have an effect on the risks or objectives of the problem or situation, there may be certain critical points at which government intervention would be most effective. There are often two frames of reference for specifying the "point" at which government intervenes in the economy or society: the specific point in time in a sequence of events and/or the particular physical target that can be distinguished from other targets. It may be necessary to revisit the objectives to determine the possible outcomes of different points of intervention.

Hazard Analysis and Critical Control Point System

The Hazard Analysis and Critical Control Point System (HACCP) is a food safety system in which critical points in the food-processing chain are identified and controls put in place to ensure that food safety hazards are eliminated. It differs from random sampling as a food safety system in that it focusses on preventing problems before they occur rather than trying to detect failures through end-product testing.

Although HACCP is an industry self-inspection program, it illustrates how government intervention points can be internalized within a program. The U.S. Food and Drug Administration, for example, notes seven steps that seafood companies are to follow to ensure seafood safety.  Each processor must:

1. identify potential hazards specific to the seafood and determine how to minimize these risks
2. identify the critical risk control points of seafood, such as cooking or storage, so as to minimize or eliminate the risk
3. establish preventive measures with critical limits for each control point
4. establish procedures to monitor the critical control points
5. establish corrective action to be taken when the critical limits for each control point are not met
6. establish means by which the system can be verified
7. establish effective record-keeping

The FDA's HACCP regulations also require seafood processors to write and comply with sanitation standards specifying the safeness of the water used in food preparation, the cleanliness of surfaces and all instruments that come in contact with the food, and proper maintenance of the food processing equipment.

In Canada, the Canadian Food Inspection Agency uses the HACCP to carry out its Food Safety Enhancement Program.

Step 4: Identify actors/institutions that can have an effect on risks or objectives

The fourth step involves identifying the actors and institutions that should be involved in addressing the problem or situation. The government's Good Governance Guidelines urge officials to explore the use of collaborative arrangements with communities, the voluntary and private sectors, and our partners in Confederation. Many of today's societal problems and issues are complex, require specialized knowledge, and are expensive to fully address. It is therefore advantageous to work with other actors and institutions to address public policy problems rather than simply taking a government-centred instrument choice perspective. Hence, it is important to bear in mind that instrument choice by government is a subset of a larger constellation of actors, institutions, instruments, and approaches that, taken together, constitute a more "distributive governance" method of addressing public policy issues. Harnessing the resources and expertise of these external actors, institutions, and processes requires a wider range of non-conventional approaches. Answering the following questions will assist officials in identifying the actors and institutions that should be involved in addressing a situation or problem.

Questions To Consider for Identifying Actors/Institutions

The diagram identifies the questions to be considered in identifying Actors/Institutions that may have an effect on risks and objectives. These are:

• Is the issue one where government can act alone?
• IS the issue on where external actors can act alone with some influence from government?
• Is the issue one where a mixed governance approach is required, that is, where government and external actors need to act together?
• What external actors operated in the same field? E.g. NGOs, policy networks, institutions, business, P/T and local governments, other federal actors, bilateral actors (Canada – U.S.), international actors (E.U., U.N.), global actors (WTO, IMF)?
• Are the external actors potential collaborators?
• Can they regulate themselves?
• Can their self-regulatory abilities be enhanced by means such as capacity and institution building?
• Are indirect forms of government action appropriate (as opposed to direct intervention through regulation)?
• Can new actors be found?
• If creating new markets, can new actors be encouraged to play a role (e.g. permit trading schemes)?

To reinforce both of the overarching rationales, depending on the case at hand, consultations with actors and institutions can be carried out at any time during the policy-making process. Because consultation itself is an iterative process within the analytical framework, it is not identified as a particular step in the process. The guidance regarding regulatory consultations with actors and institutions provided in Guidelines for Effective Regulatory Consultations, available at http://www.tbs-sct.gc.ca/ri-qr, is also useful for consultations pertaining to problems or situations in which direct regulation may not be the optimal policy instrument. Consultations encourage knowledge sharing, promote transparency and trust in government action, and can enhance public policy performance results.

Consulting key actors and institutions regarding the choice of instruments can be as important as consulting them on policy issues. When using a mix of instruments, a number of actors and institutions will likely be involved in the implementation phase. This may mean that they will need to assume new roles and responsibilities to further public policy goals. Consideration of all needs, rights, roles, and responsibilities of all of the actors and institutions at the beginning of the policy/instrument selection process can improve the quality of public policy decisions.

The following questions should be considered for effective consultations throughout the process of assessing and selecting instruments.

Questions To Consider for Effective Consultations

The diagram identifies the questions which should be considered to ensure an effective consultation process. These include:

• Which actors and institutions can influence the problem or are affected by the problem?
• What is the level of knowledge, understanding & expertise of the actors and institutions? What is their capacity to engage in the instrument choice process and provide information?
• Are the concerns and risks identified by the actors and institutions integrated into the assessment and selection criteria for instruments?
• Is consensus the goal of consultation?
• How will diverging points of view be addressed and how will the various actors and institutions be advised?
• Will there be difficult choices or trade-offs?
• Will certain actors and institutions benefit more than others?

Step 5: Considerations in assessing and selecting instruments

This step involves taking into account a number of considerations relevant to the assessment and selection of instruments. These considerations include effectiveness, legality, conformity and compliance, and accountability.

There are three dimensions to establishing the considerations for evaluating instruments. The first involves identifying the considerations that relate to managing the problems/risks found in Step I. These would include, for example, technical, legal, implementation (manageability), and conformity/compliance risks. The second dimension entails identifying and taking full advantage of opportunities for coordinating the types and/or use of instruments with other governments and agencies, both domestically and internationally. The third involves identifying considerations that reflect certain values such as economic efficiency, fairness, individual liberty, political acceptability, and accountability.

The following are considerations that are commonly used in assessing and selecting instruments.

Effectiveness: the expected capacity of the instrument or mix of instruments to achieve the intended policy objective

Will the instrument or mix of instruments do the job? Could it cause unintended or unwanted results? If it could have negative results, the policy objectives may have to be adjusted and other measures to achieve those goals considered.

Legality, individual liberty, and state coercion: the need to ensure that the government has met all legal requirements and has the authority to put the instrument into place

Legal scrutiny also includes ensuring that legal obligations have been met. These obligations are stipulated in international agreements to which the Government of Canada is a party. They include, for example, the Canadian Agreement on Internal Trade, the International Convention for the Safety of Life at Sea, the North American Free Trade Agreement, and the Agreement Establishing the World Trade Organization. A related consideration is the degree to which the use of the instrument(s) constrains the inherent legal rights of individuals. Many of the instruments available to government, particularly those that involve the use of rules, achieve their objectives by constraining the rights of individuals to engage in activities that would otherwise be permissible. A related aspect is the degree of state coercion (for example, enforcement activity) that would likely be required to implement the instrument and achieve the risk management objectives.

Conformity and compliance: the expected change in the behaviour of actors and institutions following implementation of the instrument(s)

Some types of problems, depending on their inherent risks, may require monitoring, sanctioning, and enforcement activities, while others may not. In cases of low risk or where the risks do not involve irreversible or critical harm to Canadians or the environment, it may be preferable to use instruments or mixes of instruments that encourage or promote, rather than require, a change in behaviour. Issues of consistency and coordination of policy intervention across jurisdictions and of interdependency between actors and institutions must also be considered.

The Netherlands' Table of Eleven

Maximizing conformity and compliance should be considered at the design phase when choosing a suitable mix of instruments. The Netherlands has created a checklist that helps track conformity and compliance issues throughout the policy development process. The checklist evaluates proposals against a table of 11 key determinants of compliance:

1. Knowledge of rules: target group familiarity with policies, laws, and regulations
2. Cost/benefit considerations: material and non-material advantages and disadvantages arising from violating or observing policies, laws, and regulations
3. Level of acceptance: the extent to which the target group (generally) accepts the policy, laws, and regulations
4. Normative commitment: innate target group willingness to comply or habit of complying with policy, laws, and regulations
5. Informal control: the possibility that non-compliant behaviour by the target groups will be detected and disapproved of by third parties (i.e. non-government authorities) and possibility–and severity–of sanctions imposed by third parties (e.g. loss of customers/contractors, loss of reputation) Enforcement dimensions (the influence of enforcement on compliance)
6. Informal report probability: the likelihood that an offence will come to light other than during an official investigation and may be officially reported (whistle blowing)
7. Probability of inspection: risk of an administrative (paper) or substantive (physical) audit/inspection by official authorities
8. Detection probability: the likelihood that an offence will be detected during an administrative audit or substantive investigation by official authorities (probability of non-compliant behaviour being uncovered in the event of some kind of scrutiny)
9. Selectivity: the (increased) chance of inspection and detection as a result of risk analysis and targeting of firms, persons, or areas (i.e. extent to which inspectors succeed in checking offenders more often than those who abide by the law) Sanction dimensions (the influence of sanctions on compliance)
10. Sanction probability: the likelihood of a sanction being imposed if an offence has been detected through inspection and criminal investigation
11. Sanction severity: the severity and type of sanction and associated adverse effects (e.g. loss of respect and reputation)The Netherlands' Table of Eleven

Economic implications: the expected changes to the net social benefits and corresponding distributional impacts from implementing an instrument or a mix of instruments

Consult the Canadian Cost-Benefit Analysis Guide for further details. This consideration includes not only the costs and benefits to Canadians, government, and industry, but also issues such as who may win or who may lose, and how those disadvantaged by government action will fare. Increasing the flexibility of the instrument or mix of instruments and reducing the paperwork burden can help to minimize costs. Choosing the appropriate instrument or mix of instruments can help to maximize the potential benefits of an action, whether the benefits gained are tangible (e.g. product value added) or intangible (e.g. increased safety).

Fairness: the acceptability of potential economic and social distributional consequences arising from the use of an instrument or mix of instruments on different social groups, business sectors, and/or regions of the country

The internalization of externalities should also be considered as a distributional factor (i.e. the "polluter pays" principle is applied when considering policy intervention).

Political accountability, transparency, and legitimacy: the ability of government to demonstrate the effectiveness of an instrument or mix of instruments, and the support of government departments and Canadians for the use of these instruments and their policy objectives

Jurisdictional compatibility with other instruments already in use or proposed by other governance institutions, and not only government authorities, should also be taken into account. One of the key prerequisites for accountability is transparency in both the policy development process and the administration of government programs. Transparency remains an important consideration (perhaps even more so) if the instrument mix includes measures not wholly within government control, such as industry self-regulation or delegated authority.

Determination of Legitimacy

The question of legitimacy must be confronted when considering and assessing which instrument(s) will be used to achieve public policy objectives. The values held by Canadians should be reflected in the criteria for selecting instruments. In a study for Quebec's Société de l'assurance automobile, Pierre Issalys identified and defined five criteria specific to legitimacy:

1. Political control–Parliament (to which government authority is subordinate), through its representatives, is the embodiment of the collective freedom of society.
2. Public space–The ability of Canadians to hold government responsible for its actions, either directly or through its representatives, must be facilitated and maintained by the rule of law.
3. Fairness–Government action must respect the rights of individuals and be as consistent and transparent as possible.
4. Technical–Public policy should, wherever possible, be motivated by sound empirical foundations.
5. Economy of means–Government resources should be efficiently allocated in order to meet public policy objectives.

Sample Instrument Assessment Matrix

A matrix similar to the one below can be used as an aid in assessing the suitability of various instruments.

Criteria	Importance Rating (H, M, L)	Regulatory Instruments			Economic Instruments			Voluntary Instruments			Information Instruments
		1	2	3	4	5	6	7	8	9	10	11	12
Effectiveness magnitude of risk mitigation over time speed with which risk mitigation objectives are achieved sustainability of risk management performance severity of side effects
Legality and Legal Obligations consistency with Constitution and statutes international trade agreements international social agreements
Individual Liberty and State Coercion severity of constraints on individual liberties anticipated need for application of state coercion comparative balance between constraints and enabling effects of instruments
Conformity and Compliance accessibility of rules/objectives for actors and institutions informal report probability inspection probability detection probability (from Netherlands example)
Economic Implications regulatory burden reduction of compliance costs market structure rate of cost abatement price sensitivity potential for value-added opportunities
Fairness distributional impacts across social groups, industry sectors, SMEs, regions internalization of externalities
Political Accountability and Legitimacy expected buy-in from government expected support from Canadians transparency of process and activities accessibility of information on activities and instrument performance

Step 6: Set performance indicators

Performance indicators are essential for determining whether the instrument or mix of instruments is performing as intended and is achieving and sustaining the desired risk management outcomes. Such indicators should be tentatively identified before the details of the intervention strategy are established, as they can help shape the final product. A single or small set (i.e. "dashboard") of high-level indicators should be specified to track overall risk management performance in the policy area (e.g. the number of deaths per year due to playground-related injuries and the number of serious injuries per year). Performance measures in policy sub-areas should also be identified (e.g. manufacturer defect rate for playground equipment and level of on-site adult/parental supervision). These measures will likely correspond to causal risk factors influencing the achievement of the policy goal (as identified in Step I) or will relate to the criteria used for selecting the instruments (as described in Step V). To ensure that the performance measures are meaningful (e.g. understandable, relevant, and comparable), it is valuable to engage departmental planners or evaluators in the policy process to leverage in-house expertise.

The logic model is depicted using a sample problem - in this case the projected rise in playground injuries between 1980 and 2020 - and possible measures for better performance. In the example, three measures are presented: number of deaths per year due to playground related injuries; manufacturer defect rate for playground equipment; and the number of serious injuries per year.

If a dynamic risk analysis approach is being used, it is preferable to avoid using "snapshot" performance monitoring and reporting, which may provide only a fragmented or disaggregated and time-limited picture of performance. Instead, the focus should be on tracking emerging trends in both risk-related indicators and the determinants of risk. This approach should be a foundation principle in any performance management regime.

The goal of performance monitoring and reporting should be to assess the effectiveness of the instrument(s) against the performance expectations identified in Step II and determine the reasons for any divergence. If a divergence does occur, it may be due to a problem in program implementation or in the policy development process (e.g. overestimating the power of instruments to mitigate a risk; overestimating how quickly they would take effect; or not recognizing elements of social, economic, or environmental systems that would counteract the beneficial impacts of the instruments).

The logic model established in "identifying and defining the problem" (Step I) will help to provide a sound understanding of the causal determinants of the risk so that government may be able to rely on and respond to changes in indicators relating to these factors over time (e.g. the level of awareness, compliance, and safe behaviour of the target audience). Often, these dynamics will be detectable before changes in the ultimate outcomes (e.g. a fatality or injury) due to the instruments employed.

The diagram details the gap analysis process used to identify the differences between the existing and proposed approaches through the development of an implementation strategy. The first step is to determine the appropriate instrument mix. Next a gap analysis is performed. This is followed by an assessment of resource requirements and then the development of performance monitoring and reporting. Final an implementation project plan is developed and the strategy is executed and performance monitored.

Step 7: Implement instrument or mix of instruments

Implementation involves applying the instrument or mix of instruments to manage the risks over time and achieve policy objectives. This usually requires an implementation project plan with the following components:

• articulation of an overall implementation strategy
• identification of major streams of activity
• timing of tasks, events, and major milestones
• specification of roles and responsibilities of key players
• estimation of required resources

Where the chosen intervention strategy and use of selected instruments involve a modification of existing policy and program approaches, it may be useful to first carry out some form of "gap analysis" identifying the differences between the existing and proposed approaches. The differences will largely determine the scope and content of the implementation project plan.

Annex A: Instrument Choice Frameworks

This annex summarizes instrument choice frameworks developed by federal departments and agencies. Linkages to the full documents on departmental websites will be added over time as new frameworks are developed.

Environment Canada: Qualitative Screening of Management Tools

Environment Canada's Qualitative Screening of Management Tools (QSMT) is a method for assisting in risk management decisions for considering the most appropriate policy instrument or mix of instruments to achieve public policy objectives.

The QSMT comprises questions and sub-questions under five broad criteria (e.g. environmental effectiveness, economic efficiency, distributional impacts), with a qualitative assessment (i.e. high, medium, low) conducted for each. By evaluating a range of policy instruments against predetermined criteria, the QSMT method elicits information concerning what knowledge and assumptions can be validated or refuted and identifies information gaps, policy considerations, the advantages/disadvantages of various policy instruments, and issues that may arise during consultations. The QSMT criteria and their associated questions are not exhaustive; they are indicators of key considerations, which may in turn generate further questions to add to the risk analysis.

Environment Canada has adapted the QSMT to the mandate of the department by establishing sub-questions under each of the five criteria that are designed to specify, clarify, and extract information on the appropriateness of the instrument(s).

(See Annex B for a detailed presentation of QSMT criteria.)

NOTE: This annex will be expanded over time with additional summaries of departmental instrument choice frameworks.

Annex B: Examples of Instrument Choice Frameworks

Table A-1—Criterion 1: Environmental Effectiveness

Critical question: To what degree does the instrument meet the public policy objectives?

(Note: the QSMT refers to the objective as the Risk Management Objective, which may not necessarily be synonymous with public policy objectives for the purpose of adapting it to our needs here. Public policy essentially serves to minimize the risks of a given situation or problem.)

Question	Consideration	Assessment (High, Med, Low, and variations in between)
Degree of impact 1. What is the capacity of the instrument to meet the public policy objectives?	To what degree will the instrument achieve the public policy objectives on a sustained basis? Is the instrument suitable for the life-cycle stage targeted in the risk management strategy? Does the instrument encourage pollution prevention? Would using the instrument result in any environmental co-benefits (e.g. reduction in the emission/release of another toxic substance)? Would the instrument encourage continuous improvement after the public policy objectives have been achieved?	High = instrument has strong capacity to achieve the public policy objectives Low = instrument has a low capacity to achieve the public policy objectives
Speediness 2. How quickly will the instrument achieve its maximum capacity (as identified in question 1 above)?	What length of time is required to introduce the instrument? What length of time is likely required for a behavioural change? How long will it take to achieve the instrument's maximum capacity on a sustained basis?	High = instrument requires short period of time for effect Low = instrument requires long period of time for effect
Flexibility 3. Is the instrument flexible?	As new information arises or conditions/targets change, how long is it likely to take, or how easy is it likely to be, to adjust or adapt the instrument?	High = instrument is flexible Low = instrument is inflexible

Table A-2—Criterion 2: Economic Efficiency

Critical question: In achieving the projected level of effectiveness, is use of the instrument likely to result in benefits outweighing costs, or vice versa?

Question	Considerations	Assessment
Cost Considerations 1. What are the likely incremental cost outcomes of the instrument?	What are the costs and how (in)significant are they for: the Canadian government, with respect to compliance promotion, enforcement, monitoring, administration, other; industry, with respect to compliance such as abatement, administration, adjustment and transition, training; the Canadian public, with respect to compliance (e.g. direct costs of product disposal)?	High = instrument results in significant or high costs to Canadian society as a whole Low = instrument results in insignificant or low costs to Canadian society as a whole
Benefit Considerations 2. What are the likely incremental benefit outcomes of the instrument?	What are the benefits and how (in)significant are they for: the Canadian government (e.g. reduced medical costs); industry (consider cost savings from revenue increases, productivity improvements, likelihood of positive recognition, avoided costs, or improved operating efficiency); the public (including health benefits such as decreased pain and suffering and environmental benefits)?	High = instrument results in significant benefits to Canadian society as a whole Low = instrument results in insignificant benefits to Canadian society as a whole
Flexibility 3. Does the instrument allow flexibility in decision making on the part of the targeted sector/area or stakeholder group? Note 1: Performance-based systems tend to provide more flexibility than prescriptive ones. It is important to identify your assumptions about how each instrument will be applied, as the same instrument may be used in either a prescriptive or performance-based manner. Regulations, for example, may be either performance-based or prescriptive. Note 2: Choice tends to minimize costs, as stakeholders have more flexibility in choosing the means to the desired ends.	How much flexibility do individual producers/importers/users/etc. have (per life-cycle stage targeted) to decide on the specific changes they will make to their operation, equipment, materials, purchases, etc. to achieve compliance? Does the instrument encourage competitiveness and other co-benefits? How will compliance costs change over time? Is the tool able to adapt quickly to regional, national, and international changes (e.g. new risk assessments, international protocols, new management policies, etc.)?	High = application of the instrument for this substance allows large degree of flexibility for stakeholder group Low = application of the instrument for this substance is prescriptive, with little flexibility for stakeholder group

In considering this criterion, it may be necessary to analyze costs and benefits for various sectors and stakeholders separately. For example, certain industry sectors may benefit from specific instruments, while such instruments may be cost-neutral for others and may generate costs for still others. Similarly, local governments, provincial and territorial governments, and the federal government may each experience different costs and benefits from the application of the same instrument. The distributional impact is considered within a separate criterion; the overall costs and benefits to society as a whole are the elements assessed under the Economic Efficiency criterion.

Table A-3—Criterion 3: Distributional Impact

Critical question: Would this instrument result in inequitable distribution of costs and benefits between sectors and regions?

Question	Considerations	Assessment
Distributional Considerations 1. What are the outcomes by group, region, or sector?	How are the costs and benefits distributed across regions? Are there costs and/or benefits deferred to future generations? Is there a shift in business activity and/or economic activity from one sector or region to another? Is this a desirable shift (in terms of innovation, new opportunities for Canadian businesses, or economic activity in regions of Canada)?	High = instrument results in desirable outcomes that clearly outweigh negative changes in distribution of costs/benefits of economic activity Low = instrument results in inequitable and/or disproportional distributional impacts to current and/or future generations with few desirable outcomes
Competitiveness 2. What are the competitiveness outcomes for Canadians?	Does the instrument increase or decrease competitiveness for one sector or one region in Canada? Does this instrument increase or decrease the overall competitiveness of Canada in the targeted sector or other sectors in relation to counterparts in other countries?	High = instrument results in increased competitiveness, or in decreased competitiveness that can be mitigated without unacceptable side effects Low = instrument results in decreased competitiveness that cannot be mitigated without unacceptable effects (on environmental effectiveness, market signals, or distributional consequences)
Social and Employment Considerations 3. Does the instrument result in a change in the benefits, costs, or employment of one social group more than of other groups?	Does the instrument result in a change in distribution of costs or benefits to any disadvantaged group? Are the results unfair and, if so, can they be mitigated without unacceptable side effects? Does the instrument have an impact on employment on a national or regional basis? Is the impact unfair and can it be mitigated?	High = instrument results in fair or neutral distribution of costs and benefits across social groups, or in distributional impacts that can be mitigated without unacceptable side effects Low = instrument results in unfair distribution of costs or benefits to specific social groups that cannot be mitigated without unacceptable effects
Polluter Pays Principle 4. Does the instrument reflect the "polluter pays" principle? Note: It is important to document which group or groups of stakeholders you assume to be the "polluters," where in the life cycle of the substance each tool will be applied, and how this stage is related to the "polluter" and release of the substance.	Who bears the costs associated with compliance? To what degree does the instrument assist in internalizing the health or environmental costs of the substance?	High = application of the instrument for this substance results in the polluter paying the bulk of the costs Low = application of the instrument for this substance does not place the burden of cost on the polluter

Table A-4—Criterion 4: Political and Public Acceptability and Jurisdictional Compatibility

Critical question: Would this instrument receive political and public support and be compatible with existing or proposed control measures in other jurisdictions?

Question	Considerations	Assessment
Acceptability to Current Stakeholders 1. What is the expected buy-in from the Canadian public, stakeholders, and government?	Which stakeholders (including industry, governments, and the Canadian public) are likely to support/oppose the instrument and how strong is the support/opposition estimated to be (i.e. strong support, mild support, mild opposition, strong opposition)? Does the instrument improve information available on, and public awareness of, environmental risks and benefits? Does the instrument assist in satisfying expectations of various stakeholders? Does it fail to meet expectations of various stakeholders? Is the timing of the instrument's implementation acceptable to stakeholders?	High = instrument results in support from a good cross-section of stakeholders Low = instrument results in opposition from a large portion of stakeholders
Jurisdictional Compatibility 2. Is the instrument compatible with constitutional jurisdictions, established jurisdictional responsibilities, and national environmental policies?	Does the proposed application of the instrument take into account constitutional and/or established jurisdictional responsibilities? Alternatively, is this currently an ambiguous/undefined area of responsibility? Is it compatible with existing administrative functions or are new functions required? Is the tool consistent with control measures in other Canadian jurisdictions or under development?	High = instrument is compatible with the federal government's jurisdictional responsibilities; it supports/supplements actions of other jurisdictions Low = instrument results in overlap or incompatibility with the federal government's jurisdictional responsibilities; results in new administrative requirements

Table A-5—Criterion 5: Trade and Investment Obligations

Critical question: Is the proposed instrument being considered because it is the most effective means of meeting the public policy objectives, or could it be perceived as a measure that is harmful to foreign competitors and that is being advanced primarily for non-environmental reasons (such as promoting the economic interests of Canadian firms or industries)?

Question	Considerations	Assessment
General Obligations 1. Does the instrument avoid trade restrictions and discriminatory practices (if not necessary to achieve the public policy objectives)?	Could the measure treat foreigners or foreign companies (products and substitutes, services, or investments) differently (i.e. less favourably) from Canadians or Canadian companies? Could the measure treat some foreigners or some foreign countries differently from other foreigners or foreign countries? Could the measure directly restrict the import or export of products or services, or control the methods of production used in foreign countries?	High = instrument does not result in trade complications Low = instrument could result in trade complications with one or more trading partners
Specific Obligations 2. Do the categories of measures trigger specific trade investment issues, with similar objectives as above?	For measures related to the protection of humans, animals, or plants from food, feed-borne risks, or pests/diseases, and technical regulations: Are there alternative domestic measures with a lower impact on trade that could achieve the public policy objective? (International standards are relevant to that assessment.) Is there a lack of internationally recognized scientific analysis to justify and back the measure? Investment and investors: Could the proposed measure affect foreign investment and investors by imposing requirements with regard to matters such as (but not limited to) export levels, quotas, domestic content, transfer of technology, or the nationality of senior management? Could it have an effect equivalent to an expropriation? Service: Would the measures require a service provider to have an office or be resident in Canada or require Canadian licences, qualifications, procedures, requirements, or technical standards? Subsidies: Do the measures provide subsidies that are linked to production export/import standards? Are the subsidies sector-specific and potentially trade-distorting?	High = application of the instrument does not result in any specific trade issues Low = application of the instrument raises specific trade issues