Elements of a Technical Interoperability Framework for Canadian Heritage

From wiki
Revision as of 15:03, 26 July 2018 by Jpatr029 (talk | contribs) (Added internal link)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Montréal, February 8, 2016

Report prepared by Jonathan Le Lous, Hussein Abdallah (Savoir-faire Linux), Benjamin Jean and Camille Moulin (Inno3)


EXECUTIVE SUMMARY

Savoir-faire Linux was tasked with studying existing interoperability frameworks and preparing a report on the operational challenges and opportunities of implementing an interoperability framework based on open standards within the Department of Canadian Heritage (PCH).

Interoperability design was studied by comparing existing interoperability frameworks. To that end, Quebec, European, French and British interoperability reference systems were taken as examples. Through this study, the link between IT interoperability and open standards was established by defining them according to these essential characteristics: openness and transparency of the standard's development process to all interested parties, intellectual property rights allowing the standard to be implemented using any type of software (open source as well as proprietary), and availability of the specification at little or no cost.

The use of interoperable IT has a number of advantages, such as greater independence from suppliers and greater flexibility in the enterprise architecture's design and evolution.

To understand the technological and organizational context at PCH, we met with several PCH and Treasury Board Secretariat employees at the beginning of the project. The information we gathered helped us draw up an inventory of open standards applicable to PCH, namely, network protocols and file formats (e.g. office application and multimedia).

PCH's adoption of an interoperability framework would immediately require the following actions:

  • Open standards would have to be used in existing software wherever possible.
  • Procedures and directives for the selection and use of file formats would have to be updated. Open standards must be prioritized.
  • Software selection procedures would have to be changed to account for the interoperability framework.
  • Employees would need to be made aware of the issue of interoperability and open standards.

Lastly, to benefit fully from the advantages of interoperability, the Government of Canada should implement it across the board, rather than separately in each individual department.


Introduction

As with every technological discipline, information technology is constantly evolving at an ever-accelerating pace, but it has become an integral part of so many fields that its overall nature has changed. A gradual shift in terminology has occurred: we now talk more readily about digital to refer to an area that goes beyond technologies and has entered the realms of uses - as the vast majority of communications now use this medium.

The actors involved in digital interactions have not only increased exponentially in number but have also become more diverse: individuals, users, businesses, the public, institutions and things all communicate via computer exchanges. These computer exchanges have become an essential conduit, and their overall heterogeneity is an even greater reason for interoperability, meaning the ability to make a variety of information systems communicate with one another. Since this report discusses digital interactions, the expressions "interoperability of information systems (IS)" and "interoperability of information technologies (IT) or information and communication technologies (ICT)" are synonymous.

In light of Canadian Heritage's mission to support and promote Canadian culture, arts and sport, a policy that affirms interoperability is all the more important. Canadian Heritage's activities involve countless exchanges of data with all kinds of stakeholders across Canada to organize cultural and sporting events, and to fund Canadian artists and athletes.

A lack of interoperability can lead, at best, to a breakdown in communication or operations and, at worst, to an outright loss of access to its information.


Scope, Issues and Definitions

The objective of this document is to examine the relevant issues regarding interoperability within Heritage Canada (PCH) and Canadian public administration as a whole, while also keeping cultural concerns and information system (IS) management in mind. To the extent necessary, it draws on similar and/or notable international initiatives.

For any organization, especially a government agency, this need for interoperability is at two distinct yet partially overlapping levels: internal interoperability with its own information and communication technologies (ICTs), and interoperability with the ICTs of external stakeholders (other administrations, the public, businesses, associations), whether directly or indirectly, through the availability of open data or networks of linked, open data.

Interoperabilite interne externe.PNG

Canada's Open Government initiative, which includes Open Data, Open Information and Open Dialogue, further reinforces these issues, as access to and the processing of available information relies on open (common) and interoperable formats[1]. These principles are often reiterated, whether in national policies such as the Open Standards Principles (UK)[2], or in dedicated initiatives such as Open Government Standards[3].

Interoperability levels

In terms of external interoperability, the crux of the matter is not only technical, but also based on a set of concerns at various levels. The European Interoperability Framework (EIF)[4] points to:

  • the political context in which this interoperability takes place, meaning the willingness of leaders to define priorities and provide the resources and means to achieve the interoperability levels listed below;
  • legal interoperability, which ensures the legality of data exchanges (particularly with respect to intellectual property and the disclosure of specifications). These exchanges can become complex when multiple states with different legislation are involved. The primary task, here, is legislative alignment;
  • organizational interoperability, which allows the entities involved to coordinate their data handling processes. Since these entities (different governments, for example) do not always have direct counterparts or even the same internal structures from one state to another, the relevant equivalents need to be put in place according to the different contexts;
  • semantic interoperability, which ensures that the precise meaning of exchanged information is understood and preserved. Considering that concepts can differ not only between languages but also between institutions, terminological equivalents must be clearly defined to provide a framework for exchanges between one state and another.
  • technical interoperability, which involves software interfaces and can, itself, be divided into two sub-categories, to distinguish syntactic interoperability, which concerns the format of the information to be exchanged (how data is coded and saved in a file) as opposed to exchange protocols and application programming interfaces (APIs) (sets of rules, instructions and routines that make it possible to exchange messages between systems or even run two applications jointly).

The generic definition of interoperability encompasses all of these dimensions:

"Interoperability, within the context of European public service delivery, is the ability of disparate and diverse organisations to interact towards mutually beneficial and agreed common goals, involving the sharing of information...through the business processes they support, by means of the exchange of data between their respective ICT systems."

Niveaux interoperabilite.PNG


Internal interoperability is much more focused on the technical dimension. Particular attention should be paid to its definition, to ensure that it faithfully reflects expectations associated with the term, particularly in terms of independence—the term sovereignty is sometimes used—and neutrality. A commonly used definition is the one proposed by the Interoperability Working Group of the Francophone Open Software Users' Association (AFUL)[5]:

"Interoperability is a property of a product or system, whose interfaces are completely understood, to work with other products or systems, present or future, without any restricted access or implementation."

This definition is also a reminder of the vital link between interoperability and open standards. The specified interfaces undergo standardization and/or normalization processes to ensure the emergence and maintenance of common, documented references. This normalization work is done by specialized agencies that are usually either government agencies or organizations created by professionals from a given industry sector. These organizations include the Internet Engineering Task Force (IETF), the International Organization for Standardization (ISO) and the Organization for the Advancement of Structured Information Standards (OASIS). In general, existing interoperability reference works recommend that administrations and their suppliers evaluate and use existing industrial standards in the same way as standards issued by national standardization bodies.

Open standard features

An open standard is a standard that can be implemented by everyone (in open source and proprietary solutions alike) and whose evolution is not dependent on a particular actor. The term "closed standard," on the other hand, is used when the specification is not disseminated or when it is subject to access restrictions (e.g. confidentiality agreement, non-free licences, patents). The use of open standards is necessary to ensure interoperability between products or systems.

The exact formulation of these elements can vary depending on who has jurisdiction, but there is a high level of convergence, as illustrated below by the comparison of the RGI[6] (France's Référentiel Général d'Interopérabilité, or General Guidelines for Interoperability), the CCIGQ[7] (Cadre Commun d'Interopérabilité du Gouvernement du Québec, or the Quebec government's common interoperability framework) and the EIF[8] (European Interoperability Framework). The criteria set out in the British Cabinet Office's Open Standards Principles (OSP)[9] are identical to the CCIGQ criteria.

The criteria found in the four policy documents are as follows:

  • A process for the standard's development that is open and transparent to all interested parties.
  • Intellectual property rights that allow the standard to be implemented using any type of software (open source as well as proprietary).
  • Specifications available at little or no cost.

A fourth criterion frequently included is the standard's governance by an independent body.

The CCIGQ and the OSP add a criterion that the standard must be supported by the market and demonstrate independence. The following table provides a side-by-side comparison of the relevant clauses from each policy document:

CCIGQ/OSP RGI EIF
Collaboration – the standard is maintained through a collaborative decision-making process that is consensus based and independent of any individual supplier. Involvement in the development and maintenance of the standard is accessible to all interested parties.

Transparency – the decision-making process is transparent and a publicly accessible review by subject matter experts is part of the process.

It should be developed on the basis of a decision-making process that is transparent, open and accessible to all interested parties. All stakeholders have the same possibility of contributing to the development of the specification and public review is part of the decision-making process.
Fair access – the standard is published, thoroughly documented and publicly available at zero or low cost. The functional and technical specification of the standard must be complete, public and without any restricted access or implementation.

The specification should be available at zero cost (or at least at very low or negligible cost and without any restriction on its reuse, especially in the case of open-source software).

The specification is available for everybody to study.
Due process – the standard is adopted by a specification or standardisation organisation, or a forum or consortium with a feedback and ratification process to ensure quality. It should be maintained by a non-profit-making organisation (standardisation body, forum, consortium, etc.).

A schedule of developments should be published informing interested parties of the content of the subsequent versions.

The standard is adopted and maintained by a non-profit organization, and subsequent developments are based on a transparent decision-making process.
Rights – rights essential to implementation of the standard, and for interfacing with other implementations which have adopted that same standard, are licensed on a royalty free basis that is compatible with both open source and proprietary licensed solutions. These rights should be irrevocable unless there is a breach of licence conditions. The rights of the standard should be on a rights-free basis and compatible with open-source and proprietary software. Intellectual property rights related to the specification must allow implementation in both proprietary and open source software.
Market support – other than in the context of creating innovative solutions, the standard is mature, supported by the market and demonstrates platform, application and vendor independence.

Resulting recommendations

Ultimately, the following could be considered:

  • Draw inspiration from the UK government, which used an open process[10] to select a set of open standards[11] and which developed a policy on open standards principles.[12] In addition, it provides a detailed list of questions that may be relevant to ask regarding each criterion, in order to assess the different standards.[13]
  • Define a uniform presentation and classification method for standards. France's RGI, for example, proposes using Wikipedia to describe these standards. By also adding a link to the standard's Wikipedia page, the objective is to have the most up-to-date view of the standard. (Wikipedia's articles on these standards are regularly updated.)
  • Establish "interoperability profiles," as is the case with version 1.9.10 of the RGI[14] (submitted for public comment), which introduced them to establish a clear, uniform classification for the standards, and to effectively deal with their proliferation by selecting only the most relevant ones for the prevailing circumstances. An interoperability profile is therefore "a limited body of standards to be used in a given context or usage."

Interoperability and open-source software

From a design standpoint, interoperability and open-source software are clearly separate, as are open-source software and open standards. From a cultural and historical standpoint, however, there are strong ties between these concepts. This can be explained by the "community of values": due to their open nature, open-source software economic models are based less on strategies to lock in the user and more on interoperability to benefit the user.

Furthermore, open-source software is a good indicator of the actual degree of interoperability: if a standard cannot be implemented within an open solution, then the standard cannot be considered open.[15] Conversely, some "standards" gain acceptance not through formal standardization processes, which are considered too lengthy, but through an alternative form of dissemination of libraries under benchmark open-source licences.

Very early on, Europe pointed to the close relationship between the two concepts, for example, within the European Interoperability Framework for Pan-European eGovernment Services:

"Open Source Software (OSS) tends to use and help define open standards and publicly available specifications. OSS products are, by their nature, publicly available specifications, and the availability of their source code promotes open, democratic debate around their specifications, making them both more robust and interoperable. As such, OSS corresponds to the objectives of this Framework and should be assessed and considered favourably alongside proprietary alternatives." (p. 10)

Interoperability and security

An open standard is not necessarily more or less secure than the proprietary format. Take the presentation of the French Ministry of Defence, for example, which compares OpenOffice, OpenDocument and Open XML formats.[16] The Ministry arrives at the conclusion that the OpenDocument format (.odf) is not intrinsically more secure but it makes it easier to detect and filter dubious content. In general, the security of an information system is a fundamental concern on top of, but not directly part of, the issue of interoperability. For this reason, this report does not discuss computer security per se.

Internal interoperability and enterprise architecture

Independence and substitutability

From an internal perspective, interoperability is closely linked to the issue of enterprise architecture (urbanization), in that it allows its various components to be decoupled while remaining integrated. Thus, the Quebec government's common interoperability framework (CCIGQ)[17] is linked to the Cadre de référence de l'architecture d'entreprise gouvernementale (government enterprise architecture reference framework); and version 2 of France's RGI (General Guidelines for Interoperability) refers back to the Cadre Commun d'Urbanisation du Système d'Information de l'État (common urbanization framework for the State information system).[18]

In the context of ICTs fully under your control, one option to ensure the components' integration is standardization, either of the products directly or within a family of products generally offered by a single vendor. This approach can have functional advantages (generally in terms of a smooth integration between products), but it has the disadvantage of tying internal information technologies to an external system or a particular vendor. This connection can become both a technical handicap (by ruling out the possibility of adopting new, more relevant solutions) and an economic handicap (by decreasing opportunities to negotiate and by increasing switching costs). In an ideal interoperability scenario, however, the building blocks are substitutable: each one can be changed more easily and independently. This approach strengthens the freedom of choice and makes the information system (IS) more flexible, because each building block can be replaced by another with the same functionality without impacting the rest of the IS.

Interoperability and application adherence

In an internal IS, interoperability can be viewed in two dimensions: the horizontal dimension (between two separate applications), which by nature is the dimension at play when two different ISs interact; and the vertical dimension, which concerns the components of a single application. This is typically the case between the application itself and the underlying infrastructure components (for example, an application may require a specific database that, itself, works only on a particular operating system, without any functionalities coming into play). This is where application adherence factor in, falling outside the scope of interoperability but nevertheless sharing a number of fundamental principles with it.

Interoperabilite verticale.PNG

Take, for example, the French Gendarmerie Nationale (national police). It implemented a very strict policy for the transferability of software solutions, the reversibility of outsourcing, and strict independence between applications. Therefore, no project should require one software to be selected over another. Any interface between applications must respect an open standard, and the software must be multi-platform. This has made it possible to eliminate application adherence at the operating system level. Since 2008, the vast majority of workstation changeovers were to Linux, and the transition from Windows to Linux was a non-event, with users switching effortlessly from one to the other. Since then, workstations and software versions have been standardized, generating substantial gains, including a 40% cost reduction for the lifecycle of the workstations.[19]

Interoperability and cloud computing

Cloud computing consists of providing infrastructure services and application services on demand. This is made possible through a high level of virtualization of hardware components by way of service infrastructure software (such as OpenStack). Cloud computing is based on a complex software architecture that simultaneously manages the elasticity of resources (processors, random access memory, storage and networks) and the ability of applications to make optimal use of these resources.

Clouds can be private, meaning used internally by a single company; public, meaning hosted by a third-party organization; or hybrid, meaning a combination of the two models.

In terms of interoperability, there is no fundamental difference between a conventional infrastructure and a cloud, other than the complexity of the latter and the need to standardize vertical and horizontal components to a much greater degree. In particular, any outsourcing of an IS to a public cloud must take into account the shared services' or the private or third-party organization's ability to apply standards that allow for the possibility of changing providers or reinstating a certain number of services, if necessary. In this context, the lack of an interoperability framework and standards accepted by the provider poses a significant risk to the IS's longevity.

With respect to deploying a private cloud, interoperability promotes competition between providers, with respect to both hardware and software, as well as the ability to further develop the infrastructure over time. Indeed, the use of standards, through APIs for example, increases independence from the different drivers specific to the hardware components, and makes these infrastructure services independent from the software it deploys.

The implementation of a cloud is therefore a unique opportunity to standardize these processes and agree on a common framework. The use of an interoperability framework and the use of standards—particularly in terms of virtualization—within the private cloud infrastructure makes it easier and less costly to outsource part of it to a third-party organization, be it a company or shared services, in a hybrid solution.

The concept of an interoperability framework

An interoperability framework is defined as a set of policies, guidelines, standards, rules and recommendations made by a network of actors with a view to achieving the highest level of interoperability possible.

It also describes the operating rules that govern the analysis, selection, adoption and updating of each of these elements.

To ensure the deployment and longevity of interoperable systems, it is necessary to jointly select the standards to be adopted as well as the conditions for their implementation.

This is why several European countries (including France and the UK) and Canadian provinces such as Quebec have chosen to establish interoperability frameworks. The results are compelling, as open standards are regularly and widely used, and always the first approach to be considered when new requirements emerge.

As mentioned in the various interoperability frameworks that have already been published, they are designed only to identify key standards and not to offer predefined, unique solutions (for example, in terms of which software to choose). The RGI goes as far as proposing interoperability profiles (key assemblies), but no further. The objective of an interoperability framework is, therefore, to facilitate and guide an organization's interoperability choices while also limiting the number of potential standards in order to guarantee maximum clarity.

Defining an interoperability framework

Canadian Heritage context

Interoperability at the heart of Canada's Action Plan on Open Government

The Government of Canada is embarking on a voluntary plan of openness, to be deployed at several levels, as illustrated by the various components of Canada's Action Plan on Open Government (2014).

An open government action plan like this cannot be considered independently of a solid policy on open data and standardized principles for interoperability (and, therefore, open standards). In that vein, in 2012 the UK government published the Open Data White Paper[20], presenting its famous Open Data Principles, which gradually became the heart of the government's data publication policy. They are at the centre of the recent updating of the National Information Infrastructure (March 2015), designed to be a foundation from which to select and implement different standards to promote open data and, more broadly, government strategies on digital and information technologies.

Therefore, as part of Canada's Open Government Action Plan, the Canadian government has adopted several open data principles, including the use of commonly owned standards.

The rationale is explained as follows: "For example, if only one company manufactures the program that can read a file where data is stored, access to that information is dependent upon use of that company's program. Sometimes that program is unavailable to the public at any cost, or is available, but for a fee. Removing this cost makes the data available to a wider pool of potential users. Datasets released by the Government of Canada should be in freely available file formats as often as possible."[21] The G8 Open Data Charter takes the same approach through the Usable by All principle, which consists of "releas[ing] as much data in as many open formats as possible." With respect to this principle, Canada is committed to ensuring that "all released open data are published in open formats and free-of-charge."[22]

Interoperability: A long-term issue for PCH

Interoperability to serve the PCH mission

The mission of Canadian Heritage is to promote "an environment in which all Canadians take full advantage of dynamic cultural experiences, celebrating our history and heritage, and participating in building creative communities."[23] To carry out this mission, PCH must hop on the digital bandwagon and take full advantage of everything it has to offer.

In pursuit of these objectives, Canadian Heritage's adoption of an interoperability policy is entirely appropriate, facilitating not only the mission of its officers but also the participation of users. This document therefore aims to present the different open standards that PCH can use to enhance interoperability with data provider and user information systems and to ensure the continuity of this computer data, some of which forms part of Canada’s heritage.

Data access and use

In the context of PCH, implementing an interoperability framework and applying open standards would make it possible to guarantee everyone access to content disseminated by the organization, without technological limitations or the need to purchase any technologies.

However, this choice is not without implications, and organizational constraints and techniques need to be taken into account while there also has to be an ongoing focus on the opportunity to be innovative and offer Canadians a high value-added service. Interoperability must not compromise data accuracy and quality.

Vendor independence

At the Treasury Board Secretariat's request, Canadian Heritage needs to streamline its portfolio of applications. The goal is to go from 300 to about 30 applications by combining similar systems within a single application, like the tracking systems, for example. PCH needs to assess possible solutions, but there is no clear direction on interoperability and open standards, which means that these criteria are rarely considered during the technology selection process.

In terms of standards, complying with these criteria will help PCH reduce the risk of access to the digital heritage becoming dependent on a single vendor. This situation of vendor lock-in is particularly detrimental, reducing the choices and potential of each stakeholder, increasing cost constraints, and posing risks in terms of data continuity and accessibility. The development of the application streamlining plan is, therefore, a good opportunity to introduce an interoperability framework.

Data continuity and interoperability with future systems

Library and Archives Canada indicates that "the ability to preserve and use digital information is at risk if the computer hardware and software needed to access the information are no longer available or if the format specifications are not obtainable."[24] Therefore, to ensure that the data that is important to PCH continues to be available in the long term, for PCH staff and the public alike, open formats need to be used during the production and archiving of digital content. This will make it possible to ensure, or at least promote, interoperability with future systems, regardless of the cost or existence of the software that was used to produce it.

However, to sustainably store data with the exact same quality, it is not enough to use open formats. The location and physical medium where the data is stored are also important to ensure their accessibility and re-use over the long term. Given the importance of this issue to Canadian Heritage, this issue is mentioned in this document even though it is not directly related to open standards. In a sense, it is about the interoperability of data storage media.

It is important to ensure that the servers containing PCH files belong to PCH and are hosted either on PCH premises or in a space reserved for PCH at an off-site data centre. There are a number of external file hosting solutions, such as Dropbox, Microsoft OneDrive and Google Drive. These solutions can be very practical, but the servers, and therefore the data that they store, are not under PCH's control, and the government should not trust private companies to store its data. The government cannot control these companies' practices with respect to preserving and accessing the data stored on their platforms. Lastly, these companies are generally located outside of Canada, which can sometimes cause legal problems. These solutions could, however, be used for the off-site storage of files that do not contain sensitive or highly important information.

It is therefore advisable to store data on an infrastructure that is owned by the government. Provisions must also be made to ensure that the data is stored on reliable servers containing multiple replication disks that are regularly backed up, and that a back-up copy of the most important data is stored off site, in the event of a disaster at the main site. In our meetings with PCH, we discovered that some data is stored only on one computer in just one office.

For data archived on digital media such as hard drives, CD-ROMs and DVD-ROMs, and flash drives (SSDs, USB keys, SD cards), it is important to have a long-term retention plan: these digital media have a limited lifespan that depends on the physical quality of the media and the conditions in which they are stored. They are likely to become unreadable after a few years or decades. It is therefore important to regularly verify that these media are still readable and to copy important data over to new media. Considering that this reliable strategy is also costly, PCH must clearly identify the most important data to which this strategy is to be applied.

Criteria for including standards in an interoperability framework

A certain number of criteria should be considered when selecting the relevant standards to be recommended to an organization. Ideally, an organization would use only open standards, meaning standards that meet the criteria presented in Section 1.2: developed collaboratively and in a transparent manner, with no restrictions on access or implementation (i.e. no restrictive software patents), maintained by an independent body and supported by the industry.

For a standard to be included in an interoperability framework, it is not enough for it to be open. It must also be relevant, mature, and easy to deploy. In this context, relevance refers to how well the functionalities offered by the standard match the functionalities required by the organization. This is a functional criterion. As for maturity, this refers to whether the format has a proven track record. Lastly, a standard's cost of implementation is an essential criterion.

Finally, some standards that do not meet all the criteria of an open standard are so widespread that they could still be included for the purposes of interoperability.

Functional criterion: Information transmission and retention

The first selection criterion for a digital format or protocol is related to function: does it allow information to be transmitted or retained in a relevant manner? For example,

  • Does the document format accept tables and images?
  • Can the photo or digital drawing format be printed with a given paper format or viewed on a given screen size, with a quality deemed acceptable by users?
  • Does the size of audio files containing music allow them to be quickly downloaded over a typical residential or mobile connection?
  • In the case of compression, is any data from the original file lost?
  • Can the file be easily edited/modified later?

Functional criteria are for open formats as well as closed or proprietary formats. In some cases, there may not be any open formats that meet all the functional criteria. When this occurs, it may be acceptable to use a proprietary format, as long as there is also a copy in an open format.

Replacement cost (switching cost) criterion

Naturally, cost is an essential criterion when comparing the software and solutions needed to use the different standards.

For this comparison, it is important to include migration and switching costs (the cost of replacing one IT solution with another). A study by Delft University in the Netherlands explains that the costs of replacing an IT solution with an IT solution from another vendor are high when the replaced solution uses closed formats, even if the replacement solution uses open standards. However, once an organization implements open standards, vendor replacement costs are lower precisely because the open standard solution proposed by the new vendor is interoperable with the previous solution. The advantages of using open standards are, therefore, long term once these open standards become the norm for an organization's IT.[25]

According to the same study, the following costs are lower when the solution to be replaced uses open standards:

  • Costs related to incompatibility and network effects: When the replaced solution and the replacement solution use the same open standards, incompatibility risks are minimized. Reduction of these incompatibility costs is one of the main purposes of an interoperability framework.
  • Searching costs: For an IT solution using open standards, the effort needed to find compatible replacement solutions is likely to be less than for a solution using closed formats specific to a particular company.
  • Learning costs: Two applications using the same open format (e.g. ODF for office applications or JPG for images) may have different user interfaces, so end-user learning costs are not reduced. However, when dealing with internally developed applications that use these formats, the use of an open format reduces the learning time of the developers who have to create new applications or new links between existing applications.

In summary, although the adoption of open standards does not always lead to immediate savings, it makes it possible to avoid vendor lock-in costs down the road.

Examples of interoperability frameworks

Europe

From 2004 to 2009, the European Commission had the IDABC (Interoperable Delivery of pan-European eGovernment Services to Public Administrations, Businesses and Citizens) program to facilitate the trans-European exchange of administrative information. It was extended, from 2010 to 2015, by the ISA (Interoperability Solutions for European Public Administrations) program, which was in turn followed by ISA2, set to run from 2016 to 2020.

In December 2010, an overview was published ("Towards interoperability for European public services")[26] introducing the European Interoperability Strategy (EIS)[27] and the second version of the European Interoperability Framework (EIF)—whose development was somewhat controversial.[28]

This framework is a point of convergence for the National Interoperability Frameworks (NIFs) of member countries. The NIFs are not direct transpositions of the EIF; rather, they are documents, often pre-existing, that cover these topics and that ultimately must converge to implement the EIF. They are generally more technical than the EIF and may cover cases of internal as well as external interoperability. Under the ISA program, the National Interoperability Framework Observatory (NIFO)[29] was created to provide an overview of how the various NIFs are evolving.

A parallel can be drawn between the relationship that exists between national and European interoperability frameworks in Europe and the relationship that could exist between provincial and federal interoperability frameworks in Canada. However, that is beyond the scope of this report and could be difficult to carry out on account of the political characteristics of federal-provincial relations in Canada.

France

France's General Guidelines for Interoperability (RGI)[30] are a document developed by the Interministerial Directorate for Information and Communications Systems (DISIC) to address issues of interoperability between French administrative authorities. The most recent version available is 1.9.10, and it is awaiting final validation to become version 2.0. It primarily addresses external interoperability, but specifies: "For their internal needs, the administrative authorities remain free to choose the norms, standards and practices to be implemented. However, it is desirable that they should by default follow the recommendations of the RGI."

At the technical level, it contains a list of standards as well as a series of technical profiles that correspond to consistent bodies of standards adapted to specific contexts.

UK

In March 2011, the British Cabinet Office published its Government Information and Communications Technology (ICT) Strategy.[31] This strategy has two main objectives. The first is to create a fairer and more competitive marketplace for open software, and the second is to impose compulsory open standards.

More specifically, the UK government has decided to choose the OpenDocument format (ODF) for all editable office records.[32] Considering the ubiquity of office application files within its organizations and the quantity of data saved in these files, the choice of an open standard like ODF is a major step towards greater interoperability.

Quebec

The Quebec government's common interoperability framework (CCIGQ) was designed by the Quebec Treasury Board Secretariat to serve as a standard-setting reference framework for any actor from a public body playing a role in the design, development and management of an information system. It can be used as input and reference material for government enterprise architecture and corporate enterprise architecture.[33]

This framework falls under the Framework Policy for Governance of Information Resources, which sets out the following objectives:

  • Minimize supplementary investments: Generally, open standard software can function on multiple platforms, while the use of closed formats, such as Microsoft or Adobe formats, requires software that functions only on (certain versions of) Microsoft Windows and Mac OS X, making the organization obligated to purchase licences for the operating systems as well as for the applications. More generally speaking, open standards often make it possible to use open software as well as proprietary software, whereas, by definition, it is difficult, if not impossible, to have open software that implements proprietary formats in a fully compatible and legal manner.
  • Utilize information resource as a lever of change.
  • Ensure rigour and transparency in investments.
  • Optimize the management of expertise and know-how.
  • Ensure information security.
  • Take advantage of open-source software.

Review of protocols and formats

This section discusses the informatics protocols and formats that are relevant to PCH. Although most of them are already in use, it is worth presenting them systematically and making recommendations for interoperability. The protocols and formats listed are based on the functional areas identified in our meetings with PCH employees.

The selected protocols and standards must, therefore, be functionally relevant (i.e. meeting PCH's computing requirements) as well as mature, easy to deploy, and industry supported. They must also be open as outlined in Section 1.2. However, some formats that do not meet all of the open format criteria are presented here, either as a comparison or because they are widespread. In this case, the use of these formats may be acceptable if a copy is also stored in an open format.

References used in this document

When a format or protocol is specified in an Internet Engineering Task Force (IETF) Request for Comments (RFC), in order to save space, the full URL is not provided. All RFCs have a URL in the following format: https://tools.ietf.org/html/rfcNNNN, where NNNN is the RFC number. The IETF is an international, informal group—open to any interested individual—that participates in the development of Internet standards. When the protocol or format is not specified in an RFC, the source's complete URL is provided. Many of the format and protocol summaries use the Wikipedia descriptions (either the French or the English version).

The following documents produced by the governments of France and Quebec were also used in the preparation of this document:

Network infrastructure protocols

Network layer IPv4 and IPv6
File downloading and uploading FTP, HTTP, SFTP/SSH POP3, IMAP4, SMTP
Email POP3, IMAP4, SMTP
Real-time communication SIP

Network layer

Protocol
IPv4 Original Internet Protocol. The pool of IPv4 addresses is becoming exhausted.

IETF RFC 791, updated by RFC 1349, RFC 2474, RFC 6864

IPv6 (to be used for future deployments) New version of the Internet Protocol

IETF RFC 2460

IPv4 is the original version of the Internet Protocol that, even now in 2016, forms the basis of Internet and local network communications. However, the 32-bit addresses used by IPv4 no longer allow unique addresses to be provided for all Internet users. For this reason, but also to better incorporate security and facilitate network auto-configuration, the new Internet Protocol IPv6 was designed to replace IPv4.

IPv6 uses 128-bit addresses, which resolves the issue of IP address exhaustion over the long term and prevents the need to use Network Address Translation (NAT). Although NAT helps deal with IPv4 address exhaustion in the short and medium term, it introduces a number of difficulties in applications that require point-to-point connectivity (e.g. Voice and Video over IP, instant messaging, file exchange). In addition, IPv6 offers the following improvements over IPv4:

  • Integration of IPsec for secure communications. IPsec can be used with IPv4, but it comes integrated into IPv6.
  • Easier network auto-configuration ("plug and play" capable)

Although a full migration to IPv6 is not possible at PCH and does not directly address the issue of open formats, we recommend using IPv6 for future network deployments in order to guarantee PCH's interoperability with IPv6 networks and applications. In fact, the American Registry for Internet Numbers (ARIN) ran out of IPv4 addresses in September 2015[34]. For that reason, PCH must at least ensure that all new network hardware supports IPv6. All modern operating systems, such as Windows (since Vista), Mac OS, GNU/Linux, Android and Blackberry support IPv6.

File exchange

Protocol
HTTP/1.1

HTTPS

Standard transfer protocol for web pages. Became universal with the widespread use of the web and mobile applications.

IETF RFC 7230 to RFC 7237

FTP File Transfer Protocol

IETF RFC 959, RFC 3659 and RFC 2640

SFTP (SSH) Protocol for secure network services such as Secure SHell (SSH) (command line terminal) and file transfer.

IETF RFC 4253

HTTP/HTTPS

HTTP (Hypertext Transfer Protocol) is the preferred protocol for file transfers because it does not require the use of any particular software: all you need is a web browser. Its secure version (HTTPS) is recommended whenever a website requires a password to be sent for authentication purposes or whenever files contain sensitive data. Adopting HTTP/2, which is the new version of this protocol, is not yet recommended at this stage: its use on the Internet is still low (7% of web sites were using it in April 2016[35]) and it is not yet considered to be mature enough.

In the context of an organization such as PCH, HTTP has the advantage of using just one TCP (Transmission Control Protocol) connection to standard ports (80 and 443 in general), which firewalls generally already allow on an outbound basis in order to authorize web browsing. Also, HTTP has little difficulty traversing NAT, which can be more problematic for other protocols such as FTP.

FTP

FTP is a protocol designed to exchange files between clients and servers. It is one of the oldest protocols still in use, as it predates TCP/IP. FTP is adapted for applications that require a lot of file uploads. Unlike HTTP, which requires you to develop a web form or web application for uploading files, an FTP client is all you need to send files to the server. FTP is not a secure protocol, and all transfers, including passwords, are sent in the clear. We can therefore say that FTP can be used to transfer a large quantity of data that does not need to be encrypted internally, but it is preferable to use HTTP/HTTPS to share files with the general public.

SFTP (SSH)

For internally encrypted transfers, particularly those involving Unix computers (For example, GNU/Linux, FreeBSD, Mac OS X, and other proprietary Unix systems such as Solaris, IBM AIX, HP-UX), the SSH protocol is recommended. The SSH File Transfer Protocol is sometimes referred to by the acronym SFTP (where S stands for "secure"), but it is a completely different protocol from regular FTP. In addition to allowing remote administration, SFTP allows files to be transferred securely and tunnels to be set up for any type of TCP connection. An SSH server is installed by default on most Unix systems, and there are free clients for Windows.

Email

Sending email
Protocol
SMTP

SMTPS

Simple Mail Transfer Protocol used for all email transmissions over the Internet. SMTPS refers to the use of the standard SMTP over a TLS (Transport Layer Security) connection.

IETF RFC 821, RFC 5321

Receiving email
Protocol
POP3

POP3S

Post Office Protocol 3, used to receive email on a server. It can be secured by TLS.

IETF RFC 1939

IMAP4

IMAP4S

Internet Mail Access Protocol V4, used to receive email on a server, but offering more features than POP3. It can be secured by TLS.

IETF RFC 4253

Storage format
Protocol
Maildir Format that uses the operating system's file system to store emails in separate files.

IETF RFC 1939

SMTP is the Internet standard for sending emails, and POP3 and IMAP4 are the standards for receiving emails; as such, they are supported by all email clients.

PCH has finished migrating its emails from an IBM solution (Domino + LotusNotes) to a Microsoft solution (Exchange + Outlook). Microsoft Outlook communicates with the Exchange server by using the proprietary protocol called MAPI rather than the above-listed standard protocols. Although it is possible to use IMAP4 in this scenario, users would lose other collaborative features, such as the ability to share calendars and address books. It is therefore not recommended to use IMAP4 for the configuration of Microsoft Exchange clients.

IMAP4 could, however, be used for email archiving. The Microsoft Exchange server saves content (including emails) in a proprietary format (EDB), and the Microsoft Outlook client saves them in PST format. There are software programs to extract emails from EDB files to PST files, and from PST files to EML files, but they are proprietary programs. It is preferable to extract all important emails in an open format to ensure they will always be readable, regardless of the decisions that the proprietary software publishers may make.

An archiving server that supports the Maildir format may be a storage solution. Emails would be retrieved from the Exchange server using the IMAP4 protocol. This way, even if the Exchange server is no longer there (due to technical issues or if PCH switches to a different solution in the future), it would be possible to retrieve the emails directly. In fact, the Maildir format saves each email in a file within a normal file system tree structure. In this file, the email content stays in text format, and attachments are in Base64. The Maildir format is just one storage option. Any solution that stores emails in an open format is acceptable.

Warning: By default, POP3 and IMAP4 are disabled in Microsoft Exchange Server 2013, but they can be activated[36]. If Microsoft were ever to stop supporting these protocols in Microsoft Exchange, this product would no longer meet the interoperability requirements defined in this document.

Real-time communication

Protocol
SIP Session Initiation Protocol is an application layer protocol originally developed for voice over Internet Protocol (VoIP) but also used for video conferencing and instant messaging. It provides an authentication and location service for multiple participants. It also negotiates the types of media that can be used by the different participants.

IETF RFC 3261, RFC 6665

PCH does not have a standardized telephone system. Office phones have been replaced by cell phones, but sometimes they do not work well inside buildings. This is not a data format issue because a telephone system is not used to store or retain data, but it can be an interoperability issue. An IP telephone system based on open standards like SIP and unpatented codecs can ensure interoperability between different server and terminal types (hardware, software, mobile). It also facilitates interoperability with external telephone systems, and with email and collaboration systems (calendars, instant messaging).

File formats

The following table shows the file formats that were considered in this study. The formats in red do not meet all open format criteria but their description has been included here because of their wide popularity. If dissemination in these formats may be required for the purposes of interoperability with user software/devices, it is recommended to always have a copy in an open format. Formats in italics are actually container formats: they require one or more codecs to represent content in digital format. They are commonly referred to as "MP4 files," "OGG files," etc., which is why they have been included among the formats.

Character encoding UTF-8
Compression Bzip2, Gzip, ZIP, 7z/xz, TAR
Office applications ODF, OOXML, PDF
Collaboration vCard, iCalendar
Web HTML5, CSS3, Javascript, JSON, XML
Images PNG, JPEG, SVG, TIFF
Audio Opus, OGG/Vorbis, MP3, FLAC, AAC
Video MP4, MKV, WebM, OGG/Theora, VP8, VP9, H264

Character encoding

Format
UTF-8 Computer character encoding designed to encode every writing system and alphabet in the world.

IETF RFC 3629

UTF-8 encoding can be used to encode most existing writing systems, particularly the Latin alphabets used for both of Canada's official languages, as well as alphabets used for indigenous languages, like Inuktitut. According to W3Techs, 85% of websites use UTF-8 encoding[37]. PCH has already adopted this encoding as the standard for websites. The new government portal Canada.ca also uses UTF-8. All text files produced manually or by other applications must also be saved in UTF-8 rather than in ASCII (or one of its extensions, such as ISO 8859-1).

Compression

Format
BZIP2 In general, bzip2 compresses files more effectively than zip or gzip, but to the detriment of compression speed.

See http://www.bzip.org/

GZIP gzip is a compression format based on the deflate algorithm used extensively in free operating systems like GNU/Linux.

IETF RFC 1952

ZIP Used to compress a set of files into a single (compressed) file. This is the format used in OpenDocument office application formats.

Format specification provided on the PKWARE site: https://pkware.cachefly.net/webdocs/casestudies/APPNOTE.TXT

7z / xz Compressed file format that can use the LZMA chain algorithm. Offers a higher compression ratio than gzip or bzip2.

Format information: http://www.7-zip.org/sdk.html

TAR TAR is a Unix standard used to create archive files that can be compressed by gzip, bzip2 or LZMA.

IEEE POSIX.1-2001

Most compression formats currently in use are open, so there is no risk of interoperability issues. Existing proprietary formats, such as RAR, must nevertheless be avoided. When considering the above formats, the choice is between maximum compression ratio, compression and decompression time, and memory use. For example, the LZMA algorithm used by the 7z and xz formats has a higher compression ratio than bzip2 (which, itself, has greater compression than gzip), but to the detriment of compression time. However, the decompression time of a file compressed by LZMA is in between the gzip and bzip2 times. Gzip offers the advantage of using less memory, which is important in certain applications[38].

Web

Format
HTML5 Language specifically designed for web page creation.

W3C specification: http://www.w3.org/TR/html5/

JavaScript JavaScript is a scripting programming language used mainly on interactive web pages but also for servers.

ECMA-262 ISO/CEI 16262

CSS Cascading Style Sheets is a computer language that describes the presentation of HTML and XML documents. Version 3 or higher is recommended.

W3C specifications: http://www.w3.org/Style/CSS/current-work

JSON JavaScript Object Notation is a text data format.

IETF RFC 7159

XML Extensible Markup Language designed to facilitate the automated sharing of complex content.

W3C specification: http://www.w3.org/TR/2008/REC-xml-20081126/

PCH has already adopted web standards, namely, by using the Web Experience Toolkit (WET) or Boîte à outils de l’expérience Web (BOEW), in French. The WET-BOEW is[39]: · An award-winning front-end framework for building websites that are accessible, usable, interoperable, mobile friendly and multilingual · A collection of flexible and themeable templates and reusable components · A collaborative open software project led by the Treasury Board of Canada Secretariat

The WET-BOEW is designed around HTML5, JavaScript and CSS3 to ensure interoperability with different platforms and browsers, including mobile peripherals such as smartphones and tablets. This is also one of the reasons why PDF and Flash formats were ruled out: the display is not always readable or sometimes does not work at all on mobile devices.

Office applications

Format
ODF (version≥1.2) Open Document Format is an open data format for office applications: word processing, spreadsheet, presentation, chart, graphics and database applications. It is commonly called Open Document Format (abbreviated ODF), but the official designation is OASIS Open Document Format for Office Applications.

OASIS specification: http://docs.oasis-open.org/office/v1.2/OpenDocument-v1.2.html ISO/IEC 26300-1:2015, ISO/IEC 26300-2:2015, ISO/IEC 26300-3:2015

OOXML (not recommended) Office Open XML is an ISO/CEI 29500 standard created by Microsoft, among other things, to compete with ODF. It uses the suffixes .docx, .xlsx, .pptx, etc. Only the 2013 and subsequent versions of Microsoft Office are fully compatible (for reading and writing) with this standard.

ISO/IEC 29500:2008-2012 http://www.iso.org/iso/catalogue_detail.htm?csnumber=61750 (The standard can be downloaded for a fee.)

PDF (version≥1.7) The distinctive purpose of PDF is to preserve a file's formatting, as laid out by the file's author, regardless of the software, operating system or computer used to view or print it.

Not all software programs produce PDF files in strict compliance with ISO standard 32000-1:2008.

ODF

The PCH intranet publication checklist recommends the use of Word, Excel and PDF formats, but does not indicate the exact format to be used. In our data gathering meetings with PCH, we learned that OOXML is the default format in use. Library and Archives Canada lists ODF as preferred and OOXML as acceptable[40]. ODF is also the preferred format under the Quebec government's common interoperability framework. Lastly, this format was chosen by the UK government for all editable office records[41]. What are the advantages of ODF over OOXML?[42]:

  • Supported by the most popular office application suites, such as Microsoft Office 2013 and LibreOffice. (LibreOffice is free and open. Users are therefore not required to purchase a licence to view and edit ODF documents).
  • Developed and maintained by an independent, non-profit organization (OASIS) rather than by a single private company.
  • Content (images, videos, etc.) is directly accessible, so it can be easily accessed with tools outside the office application.
  • Based on XML format: facilitates automatic extraction and processing if necessary. It is also a human-readable format.
  • An ODF document is a ZIP file that contains multiple XML files: content can be accessed directly in the event of application failure. This reduces the risk of content being corrupted or completely lost as a result of potential bugs.
  • Better accessibility (for people with disabilities).
  • Since the format is independent of any application suite, it can be used by any application that supports it.

In addition, the Office Open XML format has been criticized for its complexity (the specification is seven times longer than the ODF specification) and (being a Microsoft initiative) for the lack of openness in its development.

Since PCH is already using Microsoft Office, we recommend using ODF as the default format to ensure interoperability with other office application software (whether outside PCH or in the event that PCH uses another office application suite in the future). This way, different software would not have to be installed and users would not have to be trained on a new way to use an existing software application. PCH must ensure, however, that version 2013 of Microsoft Office is installed. Previous versions of Microsoft Office do not support ODF 1.2.

However, as indicated by the Quebec government's common interoperability framework, Microsoft's new fonts that begin with a "C" (Calibri, Cambria, Candara, Consolas, Constantia and Corbel) must be avoided in word processing contexts because they are subject to a licence that allows them to be displayed only with the Microsoft Office suite. Free fonts cannot emulate the appearance of these fonts, and none of the documents that use them will display properly, which will cause interoperability issues[43].

Open Document Format includes the following formats (with the corresponding document extensions):

File Type Extension
Word processing (text) .odt
Spreadsheet .ods
Presentation .odp
Graphics .odg
Chart .odc
Formula .odf
Database .odb
Image .odi
Text master .odm
PDF

Portable Document Format (PDF) is used to view a document in any environment while its original formatting is maintained. This is why PDF documents must comply with the standard; otherwise, they can be displayed and edited only using Adobe software. This is already an issue at PCH, particularly with the display of PDF files by certain browsers' built-in readers. The same issue occurs with other PDF forms produced by the Government of Canada that require the installation of proprietary software (Adobe Reader, of which the latest versions are no longer available for GNU/Linux).

Although PDF was uneditable in its previous versions, there are now several software applications that make it possible to rework PDF documents either in hybrid form (LibreOffice) or by editing the PDF (Inskape, Xournal).

Collaboration

Format
vCard Open format for sharing personal data (particularly in the form of electronic business cards).

IETF RFC 6350 , RFC6868

iCalendar (iCal) Standard for representing and exchanging calendar data structured in an ics file.

IETF FC 5545, RFC6868

The use of vCard and iCal formats ensures interoperability with a number of email and collaboration software applications, on both the client side and the server side.

Multimedia

This sections presents formats for image, audio and video files.

Images
Format
JPEG This format was designed for digital photography and is used to compress images to reduce the use of disk space and bandwidth, but it results in information losses. Users can choose between a higher compression ratio and better image quality. This format is supported by most digital photo devices.

W3C specification: http://www.jpeg.org/jpeg/workplan.html ISO/CEI 10918-1 ITU-T Recommendation T.81 http://www.w3.org/Graphics/JPEG/itu-t81.pdf

PNG Portable Network Graphics: This format uses lossless compression and is suited for web graphics such as logos, icons, images representing text, and images containing gradations.

IETF RFC 2083

TIFF Tagged Image File Format: This format is a container that can contain compressed or uncompressed images.

Adobe specification: http://partners.adobe.com/public/developer/tiff/index.html

SVG Scalable Vector Graphics: This is an XML-based vector image format with support for animation.

W3C specification: http://www.w3.org/Graphics/SVG/

The PCH intranet publication checklist specifies that photographs must be in JPEG format, which is an open format. The communications team also has directives on image size and resolution, but no directives on file formats. The SVG format does not seem to be in use.

SVG is recommended for web graphics that may need to be resized: it is no longer necessary to prepare multiple images of different resolutions for different screen sizes. Modern browsers support SVG[44], but a copy can be made in PNG if you wish to support older browser versions.

GIF supports only a 256-colour palette. It is therefore recommended to use PNG format rather than GIF. Unlike JPEG format, PNG format uses lossless compression, so it is better than JPEG format for saving diagrams, charts and other graphics where JPEG's lossy compression reduces readability.

Baseline TIFF does not use compression and, therefore, produces large files. The specification is owned by Adobe, so it is not an open format in the strict sense of the term, but it is not necessary to pay royalties to be able to use it. This format is often used to archive high-quality digital images, which is why it has been included in this interoperability report. However, it is not widely used on the Internet. LZW compression can be used with TIFF, as the corresponding patents expired some years ago. However, it is better to use the PNG open format when lossless compression is necessary; and if lossy compression is required to further reduce file size, it is better to use the JPEG open format.

Audio
Format
OGG This is a multimedia container that can contain audio tracks, video and text. It is a free, unpatented format.

Specification from the Xiph.Org Foundation: http://www.xiph.org

VORBIS Free, open, unpatented audio codec. Used with the OGG container.

IETF RFC 2083

FLAC Free Lossless Audio Codec: A lossless audio compression format.

Specification from the Xiph.Org Foundation: http://www.xiph.org/flac

MP3 The most popular (lossy) audio compression format. However, this format should be avoided for the preservation of musical heritage and other sound recordings because it is patented.

ISO/CEI IS 11172-3 and ISO/CEI IS 13818-3

AAC Lossy audio compression format that generally provides better quality than MP3. There are no fees for distributing and disseminating AAC content, but there are patent licencing fees for producing and developing AAC codecs.

ISO/IEC 13818-7 and ISO/IEC 14496-3

Opus Open codec designed for real-time interactive Internet applications.

IETF RFC 6716

The FLAC open format is recommended for preserving audio clips, music, and so on in their original quality. The Ogg Vorbis open format is recommended when compression is necessary, particularly to offer online audio file downloads. MP3 is not an open format, but its popularity means that it may be necessary to produce MP3 files if you want them to be readable on all devices. In this case, it is recommended to have another copy of the file in an open format.

Video
Format
MP4 Container format for audio and video content based on the QuickTime format. It is usually used with the H.264 codec for video and AAC for audio.

ISO/CEI 14496-14

OGG This is a multimedia container that can contain audio tracks, video and text. It is a free, unpatented format.

Specification from the Xiph.Org Foundation: http://www.xiph.org

H.264 Video codec commonly used with the MP4 format. Contains patented technologies.

ISO/CEI 14496-10, UIT-T H.264

Matroska (MKV) Open standard free container format that can contain audio, video and text.

Specifications: http://matroska.org/technical/specs/index.html

WebM Open format based on Matroska and designed for the web (included among the proposed formats for video support in HTML5.) WebM allows the following codec combinations:

Possible codec combinations:

  • VP8 and Vorbis
  • VP9 and Opus, or, VP9 and Vorbis

These formats, as well as H.264, are supported by the YouTube HTML5 video player[45]. Specifications: http://www.webmproject.org/docs/container/

Theora Free, unpatented video codec. Used with the OGG container.

W3C specification: http://www.w3.org/Graphics/SVG/

VP8 Video codec made open by Google.

IETF RFC 6386

VP9 Free, open, royalty-free video codec developed by Google.

https://www.webmproject.org/vp9

At PCH, there is currently a preference for Adobe's proprietary video creation format (easier to post-edit), although other formats, such as MP4/H.264, QuickTime and MKV, are accepted for receiving videos.

We recommend using open, unpatented codecs such as VP8 and VP9 for video, and Vorbis, FLAC or Opus for audio, to ensure the longevity of digitally recorded video heritage. If the Adobe format is required for editing, a copy of important videos must also be stored in an open format. That way, they can still be accessed even if the Adobe software is not available.

Summary of recommendations

The following table recaps the protocols and formats discussed in this document. Formats indicated in red are not open formats, but their popularity dictates that PCH may need to use them for interoperability purposes. However, it is recommended to always use open formats wherever possible.

Network protocols

Network IPv4, IPv6
File downloading and uploading FTP, HTTP, SFTP (SSH)
Email POP3, IMAP4, SMTP
Real-time communication SIP

File formats

Character encoding UTF-8
Compression Bzip2, Gzip, ZIP, 7z/xz, TAR
Office applications ODF, OOXML, PDF
Collaboration vCard, iCalendar
Web HTML5, CSS3, Javascript, JSON, XML
Images PNG, JPEG, SVG, TIFF
Audio Opus, OGG/Vorbis, MP3, FLAC, AAC
Video MP4, MKV, WebM, OGG/Theora, VP8, VP9, H264

PCH is already using many of the above standards, but it must pay particular attention to these cases of commercial format and protocol use:

  • Use of Microsoft Exchange and Microsoft Outlook with the MAPI protocol for email. It must be ensured that emails are archived in an open format. Activation of the IMAP4 protocol allows an archiving server to be installed (for example, in Maildir format).
  • Use of OOXML format for office applications with Microsoft Office. It is better to use ODF as the default format, especially since the latest versions of Microsoft Office support this format.
  • Use of proprietary formats for video editing: If the use of this software is indispensable, copies absolutely must be made and retained in an open format.
  • Use of non-standard PDF: Documents must be saved in standard (ISO-compliant) PDF without Adobe's proprietary extensions, or another format must be used.
  • No telephony standardization. If a new telephony solution is being considered for PCH offices, we recommend standardizing around an open protocol, such as SIP.

PCH interoperability framework impact analysis

For PCH, implementing an interoperability framework based on open standards would make it possible to guarantee everyone access to the data, information and services offered by the organization, without technological limitations or the need to purchase any technologies. From an internal perspective, interoperability is closely linked to the issue of enterprise architecture, in that it allows its various software components to be decoupled while remaining integrated. Technical interoperability can only be achieved through the use of open standards. The advantages of open standards have been outlined in Section 2 of this report.

More specifically, the adoption of an interoperability framework at PCH would immediately require the following actions:

  • The configuration of existing software would have to be revised to use open standards wherever possible.
    • Office 2013 can be configured to save in Open Document Formats (ODF) and to generate standard PDFs.
  • Procedures and directives for the selection and use of file formats would have to be updated.
  • Software selection procedures would have to be changed to account for the interoperability framework. This includes solutions selected by workstation engineering and architecture teams.
    • Software programs that fully support open standards are preferred.
    • It is important to include migration and switching costs in the total cost assessment of a software program.
  • Employees would need to be made aware of the issue of interoperability and open standards. Employees must not perceive interoperability policies and directives as just another administrative requirement to disrupt their work.

In the longer term, PCH's adoption of an interoperability framework based on open standards would help cut vendor lock-in costs and make the enterprise architecture more flexible. Indeed, the use of open standards, and the independence and substitutability of solutions makes it possible to replace one component or system with another at a lower cost than with proprietary formats. Legacy application issues are avoided because each system must be able to communicate through interfaces that use open standards. This includes cloud systems, which, even though they are not hosted on PCH infrastructure, can easily be brought back in-house or switched to another host, whether it is with Shared Services or another provider.

PCH's use of open standards would facilitate the publication of open data whenever necessary. PCH would no longer need to convert such data from a restricted format to an open format before publishing them on the open data portal.

Because Shared Services Canada and the Treasury Board Secretariat are focusing little attention on issues surrounding interoperability and open standards, except for open data, PCH could be a leader in the development of an internal technical interoperability framework for the Government of Canada.

Conclusion

The aim of this report was to introduce PCH to the concept of technical interoperability. Interoperability was defined as follows: "Interoperability is a property of a product or system, whose interfaces are completely understood, to work with other products or systems, present or future, without any restricted access or implementation." Further to the requirement of "understood interfaces," open standards were presented and defined based on Quebec, European, French and British interoperability reference systems. Open standards are, therefore, characterized by:

  • A process for the standard's development that is open and transparent to all interested parties.
  • Intellectual property rights that allow the standard to be implemented in any type of software (open source as well as proprietary).
  • Specifications available at little or no cost.

The advantages of interoperable IT systems were then presented, such as greater independence from vendors and greater flexibility in the enterprise architecture's design and evolution. A number of related topics were also discussed in this report, such as security, data continuity, and interoperability within cloud IT infrastructures.

Lastly, the open standards applicable to PCH were presented, namely, network protocols and file formats (e.g. for office application and multimedia). PCH already uses a number of open formats, but there are some adjustments to be made, particularly in terms of office application and multimedia formats.

This report is therefore a first step towards interoperability at PCH. To effectively implement an interoperability framework, both technically and procedurally speaking, everyone at PCH would have to be on board. However, to benefit fully from the advantages of interoperability, the Government of Canada should implement it across the board, rather than separately in each individual department.

References

  1. http://open.canada.ca/en/open-data-principles
  2. https://www.gov.uk/government/publications/open-standards-principles/open-standards-principles
  3. http://www.opengovstandards.org
  4. http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf
  5. http://interoperability-definition.info/en/
  6. http://references.modernisation.gouv.fr/interoperabilite
  7. http://www.tresor.gouv.qc.ca/fileadmin/PDF/ressources_informationnelles/architecture_entreprise_gouvernementale/AEG_3.1-CCIGQinteroperabilite.pdf
  8. http://ec.europa.eu/isa/documents/isa_annex_ii_eif_en.pdf
  9. https://www.gov.uk/government/publications/open-standards-principles/open-standards-principles#open-standard-definition
  10. https://standards.data.gov.uk/
  11. https://www.gov.uk/government/publications/open-standards-for-government
  12. https://www.gov.uk/government/publications/open-standards-principles/open-standards-principles
  13. http://standards.data.gov.uk/core-assessment-questions
  14. http://references.modernisation.gouv.fr/sites/default/files/Referentiel_General_Interoperabilite_V1.9.10.pdf
  15. https://opensource.org/osr
  16. http://www.decalage.info/files/PacSec06_Lagadec_OpenDocument_OpenXML.pdf
  17. http://www.tresor.gouv.qc.ca/ressources-informationnelles/architecture-dentreprise-gouvernementale/standards-et-normes/cadre-commun-dinteroperabilite/
  18. http://references.modernisation.gouv.fr/sites/default/files/Cadre%20Commun%20d'Urbanisation%20du%20SI%20de%20l'Etat%20v1.0_0.pdf
  19. https://www.s2lq.com/sites/s2lq.local/files/Retex%20-%20Quebec%202014.pdf
  20. https://data.gov.uk/sites/default/files/Open_data_White_Paper.pdf
  21. http://open.canada.ca/en/open-data-principles
  22. http://open.canada.ca/en/g8-open-data-charter-canadas-action-plan
  23. http://pch.gc.ca/eng/1310495889171/1310496202608
  24. http://www.bac-lac.gc.ca/eng/services/government-information-resources/guidelines/Pages/guidelines-file-formats-transferring-information-resources-enduring-value.aspx
  25. http://repository.tudelft.nl/assets/uuid:391a61e9-cdf4-476d-9e97-070b733450ff/295829.pdf
  26. http://ec.europa.eu/isa/documents/isa_iop_communication_en.pdf
  27. http://ec.europa.eu/isa/documents/isa_annex_i_eis_en.pdf
  28. http://fsfe.org/activities/os/eifv2.en.html
  29. https://joinup.ec.europa.eu/community/nifo/home
  30. http://references.modernisation.gouv.fr/interoperabilite
  31. https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/85968/uk-government-government-ict-strategy_0.pdf
  32. https://www.gov.uk/guidance/open-document-format-odf-guidance-for-uk-government
  33. http://www.tresor.gouv.qc.ca/fileadmin/PDF/ressources_informationnelles/cadre_commun_interoperabilite.pdf
  34. https://www.arin.net/resources/request/ipv4_countdown.html
  35. http://w3techs.com/technologies/details/ce-http2/all/all
  36. https://technet.microsoft.com/en-us/library/jj657728(v=exchg.150).aspx
  37. http://w3techs.com/technologies/overview/character_encoding/all
  38. http://tukaani.org/lzma/benchmarks.html
  39. http://wet-boew.github.io/v4.0-ci/index-en.html
  40. http://www.bac-lac.gc.ca/eng/services/government-information-resources/guidelines/Pages/guidelines-file-formats-transferring-information-resources-enduring-value.aspx
  41. https://www.gov.uk/guidance/open-document-format-odf-guidance-for-uk-government
  42. http://www.opendocumentformat.org/features/
  43. http://www.tresor.gouv.qc.ca/fileadmin/PDF/ressources_informationnelles/architecture_entreprise_gouvernementale/AEG_3.1-CCIGQinteroperabilite.pdf
  44. http://caniuse.com/#feat=svg
  45. http://www.youtube.com/html5