Technology Trends/Cloud Management Platform

Technology Trends / Cloud Management Platform Tendances Technologiques / Platforme de Gestion d'Infonuagique
200px
Status	Translation
Initial release	September 18, 2019
Latest version	February 14, 2020
Official publication	Cloud Management Platform.pdf
This page is a work in progress. We welcome your feedback. Please use the discussion page for suggestions and comments. When the page is approved and finalized, we will send it for translation.

Cloud Management is the process of evaluating, monitoring and optimizing cloud computing based solutions and services to produce the desired efficiency, performance and overall service level required.

Hide Detailed View

Business Brief

Cloud Management solutions, known as Cloud Management Platforms (CMP), help in cloud optimization, storage allocation, management, and delivery of computing services. Deployment models such as private, public, hybrid, and community cloud cannot be simply handled and managed with virtualization alone. Due to the ever increasing complexity of cloud storage, enterprises are onboarding CMP solutions in order to manage the deployment models currently in use as well as to manage the integration of deployments they intend on adopting.

The Cloud Standards Customer Council (CSCC) and Gartner define requirements and capabilities for a product to be classified as a CMP. These requirements state the functional categories that should be included: Service Management, Resource Management, Financial Management, General Services, Systems Integration, Governance and Security.

A Cloud Management Platform (CMP) is a suite of integrated products and software tools that provide for the management of public, private, and hybrid cloud environments. This includes integrated software tools to provide governance, life cycle management, brokering and automation for managed cloud resources across functional areas such as: provisioning and orchestration; service request management; inventory and classification; monitoring and analytics; cost management and resource optimization; cloud migration, backup and disaster recovery; and identity, security and compliance. Although an organization can use a CMP exclusively for a private or public cloud deployment, these toolsets commonly target hybrid and multi-cloud models to help centralize control of various cloud-based infrastructures.

Basic CMP product characteristics include incorporating self-service interfaces, provision system images, enable metering and billing, and provide for some degree of workload optimization through established policies.

More advanced product offerings include the basic characteristics, but may also integrate with external enterprise management systems, include service catalogs, support the configuration of storage and network resources, allow for enhanced resource management via service governors and provide advanced monitoring for improved “guest” performance and availability.

Technology Brief

Cloud management software is typically deployed as a virtual machine (VM) into an existing cloud environment whether it is on-premise or using SaaS. The application server, which usually contains the web interface and the software itself, relies on a database server to store the information and the data that is collected from the different cloud environments through an API (Application Programming Interface).

When a CMP is deployed on-premise it uses the client infrastructure resources (compute, storage, network, etc) to run in opposition of a SaaS solution, where the software is running in the cloud and uses the vendor resources to accomplish the same task.

A CMP provides broad cloud management functionalities atop both public provider platforms and private cloud platforms. CMPs manage cloud services and resources that are distributed across multiple cloud platforms. Depth of functionality and broad cross-platform consistency are two major factors for considering a CMP.

CMP solutions vary based on service and business requirements, as well as deployment model such as public cloud, private cloud and hybrid and community cloud. Platform specific tools are needed to leverage unique native functionality inside cloud platforms. The growing acceptance of public cloud and increased multi-cloud usage is driving the need for consistent cross-platform management.

One of the primary roles of a CMP is to provide a consolidated control plane for IT operations integration of existing application lifecycle tools, hypervisors, and cloud platforms. CMPs must integrate with internal and external systems to manage multi-cloud services. The ability to support both published APIs and provide for customization is a critical capability.

There are two deployment/hosting models for CMP, on-premises or a Software as a Service (SaaS ) offering. Some CMPs are offered for on-premises installation or for deployment by the customer within a cloud service. Others may be offered as a SaaS run by the vendor. The choice will impact the total cost of ownership, skill requirements, network connectivity profile, and ability to directly control portions of your cloud infrastructure including service level agreements.

Industry Usage

The global CMP market, in 2018, was valued at USD 8,182.2 million and is expected to reach USD 26,767.0 million, a CAGR (Compound Annual Growth Rate) of 18.4% during the forecast future period. (Market Watch, 2019) Whereas the Multi-Cloud Management market size is expected to grow from USD 1,169.5 million in 2017 to USD 4,492.7 million by 2022, at a CAGR of 30.9%.

CMP market growth has been attributed to enterprise needs and demands for greater control over IT spending and usage, help provide a surge in adoption of heterogeneous and multimodal IT service delivery environments, rapid deployment of virtualized workloads, and improved operational efficiency.

However, growth is slowed due to insufficient technical expertise and the rising security concerns for in-house development of platforms. The average enterprise uses some combination of five or six different cloud environments, typically a mix of a private on-premises and public environments. Cross-Platform CMP is becoming a key factor in simplifying and consolidating the management tools.

Cloud computing simplifies the acquisition of many services, amplifying the need for integrated CMP services that help to continually monitor and optimize the benefits realized from Cloud Services, while proactively managing risks.

As IT further permeates all types of industry, organizations turn toward adopting and utilizing the cloud for their operations. Once enterprises seriously adopt the cloud, they then tend to integrate CMP tools into their respective operations to manage this new environment. Exact CMP options are dependent on the vendor, but they all essentially assist in the management and deployment of cloud environments.

Most industries using CMPs purchase it from third party vendors. Some of the most popular cloud management platforms being used by industries are: Cisco Cloud Center, BMC Cloud Lifecycle Management, Morpheus and IBM Cloud Orchestrator.

These CMPs are helping various organizations across many industries automate cloud tools provisioning, integrate service management, manage their cloud environment (i.e., cloud resource consumption, monitoring etc.), security and more.

Canadian Government Use

In the summer of 2016 the Government of Canada (GC) published the GC Information Technology Strategic Plan (GC ITSP) and the GC Cloud Adoption Strategy, known as the “Right-Cloud-Adoption-Strategy”. The GC Cloud Adoption Strategy promoted a series of adoption principles for GC Chief Information Officers (CIOs) to consider when choosing and using IT services. This included considerations on where cloud could benefit departments and when cloud was an appropriate deployment model. The onus was on each department’s CIO to demonstrate which deployment model was right for their business context, and cloud solutions were not necessarily the default options for deployment.

In 2018 the GC Cloud Adoption Strategy was updated by the Treasury Board of Canada Secretariat (TBS) from a “Right-Cloud-Adoption Strategy” to a “Cloud-First-Adoption-Strategy”. The Cloud-First-Adoption-Strategy ensured that cloud is the preferred option for delivering IT services with public cloud being the preferred option for cloud deployment.

The TBS Directive on Management of Information Technology sets the technology architecture solutions whereby a “Use Cloud First” is the default choice in an order of preference of: 1) Public Cloud; 2) Hybrid Cloud; 3) Private Cloud; and 4) Non-Cloud (on-premises) solutions.

The “Use Cloud First” perspective change recognizes that cloud remains the preferred option for IT delivery, with Public Cloud being the preferred model for all cloud deployments. In a public cloud model, GC organizations share secure tenancy with other consumers of a cloud service, including private companies, non-profits and individuals.

Treasury Board of Canada Secretariat (TBS) is responsible for GC enterprise governance, strategy and policy for cloud services, including oversight and risk assessment of cloud service requests from GC departments. SSC is responsible for providing a light-touch cloud-brokering service by implementing contracts with cloud service providers and thereby enabling departments to use a self-service model for provisioning and managing cloud resources (for example, compute, storage, platforms). Public Services and Procurement Canada (PSPC) may also implement contracts for cloud services. PSPC will work closely with SSC to leverage PSPC’s capabilities and to collaboratively build contracting terms and security requirements.

SSC, as cloud broker, is in the process of procuring CMPs for use in the GC Cloud. The CMPs will effectively integrate with SSC’s Client Relationship Management (CRM) to help automate and manage the cloud comprehensively and efficiently.

SSC’s Cloud Services sub-roles (Cloud Broker, Enabler, and Provider) also need to utilize CMP services. Specifically, CMP supports the Cloud Broker (e.g. forecasting, monitoring), Cloud Enabler (e.g. advising, optimizing) and Cloud Provider (e.g. invoicing, metering, capacity management).

The SSC CMP strategy is based on international standards, using an extensible CMP framework, largely founded on collaborative work under the Object Management Group – Cloud Working Group “Practical Guide to Cloud Management Platforms”.

The GC Cloud Services Procurement Vehicles are already established for “unclassified” data, and in 2019 the procurement vehicle for Protected “B” will be in place, dramatically increasing the usage of Cloud Services. SSC CMP provides the governance and management required to: reduce cost overruns, support asset management, assure regulatory and policy compliance, managing security incidents and a degradation in user experiences.

Implications for Government Agencies

Shared Services Canada (SSC)

Value Proposition

A CMP plays an essential role to enable resource visibility, simplify cloud management, and optimize the utilization of resources in a multi-cloud environment.

The strengths of many CMPs vary depending on the vendor, however core CMP capabilities provide: Cost Insights and Efficiencies; Resource Management and Automation; Improved Governance and Security; and Providing Integration.

Cost Insights and Efficiencies

The primary reason enterprises implement CMPs is to manage resource usage and reduce overall cost. A CMP provides cost efficiencies by automating client consumption tracking and resource spending.

Organizations using CMPs can access and analyze cloud usage and financial information to determine applicable Broker chargeback fees to clients/partners. CMPs provision improved visibility into costs across the cloud service environment via automated cloud management policies and tasks.

Resource Management and Automation

A CMP platform provides visibility to Cloud resource management of virtual resources (application, server, storage, and network) and delivers services on-demand when needed. CMP resource management and automation capabilities include: discovery applications, servers, storage, networks/ connectivity, and services residing within both Public and Private Cloud environments, while maintaining an accurate inventory of services and assets on an ongoing basis.

Cloud customers can leverage these products to: Manage their Cloud usage; Manage the estate of deployed VMs and containers; Manage sets of services (e.g., databases);

Provide access to all of cloud resources — public, private, and hybrid from a single console.

A CMP platform is based on automatic orchestration; for example, it automates the process of finding spare resources on the network. Instead of waiting for the IT team to discover wasted resources, the CMP automatically fixes resource usage problems while continuing to monitor the performance of the cloud-based resources 24/7 to improve productivity and user experiences. The CMP increases efficiencies by automating tasks, such as spinning down of underutilized resources.

Accurate, real-time analysis and reporting of consumption along with predictive analytics is required to control costs and optimize the usage of Cloud Services. Resources that are not being used will be a drain on the organization’s finances. By tracking consumption and resource spending, a CMP will ensure that every resource is being efficiently utilized, and identify the amount of wasted resource spending.

Improved Governance and Security

Governance implies control and oversight by the organization over policies, procedures, and standards for application development and information technology service acquisition, as well as the design, implementation, testing, use, and monitoring of deployed or engaged services. A CMP platform manages Hybrid Cloud Services in accordance with an organizations policies.

Compliance refers to an organization’s responsibility to operate in agreement with established laws, regulations, standards, and specifications. Various types of security and privacy laws and regulations exist within different countries at the national, state, and local levels, making compliance a potentially complicated issue for Cloud Computing.

CMPs are designed to help with security and compliance by automatically orchestrating change processes, enforcing standardized configurations, and applying policy-based governance to workloads. IT process and procedures provide corporate governance and safeguards. CMPs provide provisioning automation by orchestrating overall approval, deployment, and stage management processes. This provides frictionless consumption for DevOps engineers, while at the same time ensuring adherence to corporate IT governance policies.

Providing Integration

A CMP’s capabilities provide the integration with internal and external systems to manage Multi-Cloud Services. The ability to support both published APIs and provide for customization (including middleware) is a key capability. Helping integrate and exchange data within CMP elements and other enterprise management services is key for cloud services to work efficiently and flexibly. Flexible integration increases an organization’s ability to share data between existing systems both within a CMP itself and between a CMP and other enterprise applications to automate workflows.

A CMP, or a collection of synchronous CMPs, support the configuration and deployment of applications to the target cloud environment. Cloud service customers, can gain the visibility of the network and assets, control and monitor their specific resource usage and operation of the cloud services that they are using by leveraging CMPs capabilities.

Challenges

Deploying, integrating and maintaining a CMP solution isn’t as simple as it sounds. Businesses want to take advantage of the benefits the cloud can offer, however they still need to manage their cloud operations and resources even with a CMP. Integrating a CMP, or suite of CMPs, across a multitude of environments and varying legacy models can be extremely challenging.

Organizations will need to understand that a CMP will greatly amplify the need for well-thought-out governance procedures and compliance measures. While CMPs can help simplify existing governance management, it doesn't alleviate the need for pre-established governance models to be in place already on the network. Considering the wide availability of Cloud Computing services, and the lack of organizational governance controls over employees engaging with such services, governance can be a source of problems.

CMPs and cloud services are still evolving, it is too optimistic to assume that a perfect solution or one specific CMP exists for all enterprises. While single-view CMP offerings can provide capabilities and insights across multiple cloud environments, with the added advantages of consolidation, they also have limitations regarding functionality and support across all hosting, deployment and service models.

The value of CMPs is in delivering the maximum level of consistency between platforms without compromising depth of functionality. Organizations will face the challenge of balancing the competing requirements of depth of functionality with cross-platform and cross-environment consistency in selecting the appropriate CMP or suite of CMPs.

Lastly, the challenge of having the right staff with the right skills is a major challenge for organizations who require CMPs to be customized for their particular enterprise architecture.

Considerations

Utilizing Cloud Computing without adequate management, oversight, and governance is poor IT practice, the risks to the networks, data centres, and data itself are tremendous. CMPs can help effectively and efficiently manage cloud services across providers and deployment models.

A CMP supports SSC in its role as Cloud Broker – in forecasting and monitoring, as Cloud Enabler – in advising and optimizing, and as Cloud Provider – in invoicing, metering, and capacity management.

SSC should be cautious determining the possible long-term ramifications of procuring CMPs instead of a subscription of services. While most CMPs provide integration with many of the same cloud service providers and private cloud infrastructures, SSC should pay close attention to the fact that some notable vendor differences exist, which can influence a procurement decision. Vendor lock-in is a major concern for CMPs and organizations can possibly find themselves in a situation where a procured CMP may work but may require some customization not permissible under the procurement contract agreement.

SSC should be aware that the CMP market is large, complex, and ever-changing. Some products are directly available from Cloud Service Providers (CSPs) or from their catalogue offerings via their partners. Other products focus directly on supporting the management of Hybrid-Clouds and multi-Clouds, and are separate offerings that may or may not be offered within a CSP catalog. Other products call themselves CMP products, but really support the goals of CMP (e.g. service management, reporting), but were never designed and architected with a primary objective of being a CMP product. CMPs focus on providing broad functionality across the cloud management domain, but their functionality scope is increasingly being challenged by the quick pace of innovation of hyper scale public cloud providers. When purchasing and deploying a CMP, SSC should have the plan and strategy to handle the imbalanced development between CMP and the cloud technology.

One single product will not support all potential activities within a CMP category across all CSPs, service models and deployment models required to respond to the GC CMP requirements.

One CMP product often will provide capabilities that support multiple CMP categories, this has several advantages (easier integration between categories, can be cost effective, simplifies administration), however, it also has associated concerns (lock-in to CMP and limited or limited capabilities within a category) that should be analyzed.

Multiple product “best of breed” CMP strategies provide a good compromise for many organizations, including: early time-to-value, cost-effective (less customization & specialized), and satisfy targeted Government of Canada Cloud priorities. CMPs must be planned and coordinated to avoid “CMP sprawl” and enable efficient integration and interoperability as needed.

As a Cloud Provider and Cloud Broker to the GC, SSC will benefit from choosing the right CMP since SSC is in the position to broker the public clouds to GC and to operate the private clouds on Premise. SSC will manage and control the cloud resources from different public providers with different cloud platforms and the consistent and consolidated management platform becomes a key consideration. SSC can leverage broad cross-platform capabilities and deep platform-specific functions in each cloud platform considering balanced requirements of efficient consistency across different cloud platforms with access to different native functionalities within an individual cloud. The CMP will effectively integrate with SSC’s Client Relationship Management (CRM) to help automate and manage the cloud comprehensively and efficiently.

Security management of Cloud Services should be managed in accordance with GC policies. SSC provides and enables secured connectivity, encryption/ tokenization, and identity credential and access management (ICAM). Creation of consistent governance procedures across cloud environments to improve security, compliance, and adherence to best-practices is a major consideration.

SSC must ensure an appropriate CMP strategy is founded on industry and public standards, within a solid CMP framework, drawing from such foundational best practices, including: Object Management Group’s – Cloud Working Group Practical Guide to Cloud Management Platforms (Cloud Standards Customer Council, 2017); the GC Cloud Computing Security Risk Management Approach and Procedures; the GC Right Cloud Selection Guidance; the GC Cloud Computing Adoption Strategy; the ITSG-33 – IT Security Risk Management: A Lifecycle Approach; and the Shared Services Canada – Cloud Service Broker Concept of Operations (ConOps) – October 3, 2017.

SSC will require a diverse skill set to deal with the array of tools across multiple functional categories and many cloud platforms to effectively leverage CMPs capabilities in functional categories and cross-platform consistency. CMPs are change enablers, where unique skills and expertise are required in order to effectively and efficiently audit, consume and Broker Cloud Services. Plans need to be carefully developed and coordinated, including skills training, processes, workflows and culture, so that incremental sustained progress can be made to successfully manage and realize the benefits from Cloud computing.

References