Managing the Data Management Maze in Hybrid and Multi-Cloud Environments
If your company is like most, it has multi-cloud or hybrid cloud aspirations. But how do you manage all your data in a secure and cohesive manner when it’s spread across multiple systems and locations? That’s a topic that is garnering intense interest across multiple sectors.
The cloud is nearly ubiquitous at this point. Almost 90 percent of enterprises surveyed in O’Reilly’s recent Cloud Adoption in 2020 report state they have adopted cloud computing, while just 10 percent say they don’t use any form of cloud. Amazon Web Services is the big dog in cloud computing, accounting for 67 percent of the market, followed by Microsoft Azure at about 50 percent and Google Cloud at about 33 percent, O’Reilly’s data shows.
All the clouds provide an array of tools for managing data within their own clouds, not to mention the huge array of applications (including data science and machine learning frameworks) for doing interesting stuff with that data. But for various reasons few companies are content to stay with just one cloud provider, in part because there's a need for data and application resilience and vendor lock-in avoidance, among others.
According to Flexera’s State of the Cloud 2020 report, 93 percent of enterprises have a multi-cloud strategy while 87 percent have a hybrid cloud strategy. The average company currently uses 2.2 public and private clouds at the moment, showing that multi-cloud is not an aspiration but a current reality.
Source: O’Reilly Cloud Adoption 2020
Unfortunately, the IT industry is still in the early days of feeling out the needs, wants, challenges and opportunities associated with managing data that is spread across multiple clouds and on-prem systems. For that reason, the words and concepts that are used are not necessarily the same.
Depending on how you approach the problem, you will use different tools and techniques. For example, the public clouds have each launched initiatives to extend their cloud-based services to on-prem initiatives. Google Cloud Anthos, Microsoft Azure Stack Hub and AWS Outposts all provide mechanisms for allowing customers to run their cloud applications on-prem.
But multi-cloud and hybrid cloud can take many forms. According to Flexera’s report, siloed applications running on different clouds is the most common form of multi-cloud, accounting for 55 percent of multi-cloud uses, but it’s far from the only way that companies do it. About 40 percent consider data integration between clouds as “multi-cloud,” followed by workload mobility, DR/failover, applications spanning multiple clouds, workload bursting, and others.
Source: Flexera State of Cloud 2020
Clearly, there are many ways to implement a multi-cloud or hybrid cloud strategy. And when it comes to managing the data residing across clouds and on-prem systems, there are a plethora of options.
According to the Evaluator Group, there’s considerable diversity in how companies can approach the problem.
“Perhaps equally as diverse is the landscape of vendors rallying around addressing multi-cloud data management requirements,” the company wrote in a recent report. “To date, this charge has largely been led by backup and data protection vendors that have identified an opportunity to leverage their metadata catalog for the enterprise’s information assets to extend into additional use cases.”
In its MultiCloud Data Management study, Evaluator Group found that 54 percent of respondents noted using backup storage system/cloud services for multi-cloud data management (tying archiving solutions as the most popular), and 48 percent reported using backup software.
It’s not surprising that backup and recovery software vendors may be leading the way in serving customers multi-cloud data management needs. After all, backup is one of the core elements of a data management strategy. But it’s not the only element. Security is a big part of data management, as well as compliance, governance, archiving, and data visibility.
The problem is, data management is a terribly broad field, and means different things to different people at different stages of the data lifecycle. Whether it’s collecting and landing data in a database, a data lake, or a data warehouse; using data to serve customer-facing applications; or using big data sets to train machine learning algorithms, data – and data management – is a core asset that’s intrinsic to a company’s success in the digital age, on the cloud and off.
Gartner says that, while multi-cloud deployments are now the norm in most enterprises, customers should be careful about expecting too much. “Cloud application portability is a worthwhile goal, but it must be tempered with pragmatism,” the company says in its 2020 Planning Guide for Data Management report, which was published in October 2019.
“It is difficult to achieve cloud application portability, and cloud database portability is even harder,” Gartner says. Running production databases in containers – which is typically viewed as a prerequisite for multi-cloud portability – works for dev and test use cases, but “production workloads for enterprise-class, mission-critical operational databases running in containers are not mainstream for enterprise use cases as of today.”
Source: Evaluator Group’s Multi-Cloud Management Evaluator Guide
The good news is that nearly every vendor in the data management these days is embracing the new multi-cloud and hybrid-cloud reality. From databases and catalogs to data fabrics and backup tools, there is broad recognition among vendors that companies expect their offerings to work in multi-cloud and hybrid-cloud scenarios. This is what Gartner refers to as the “Cloud Data Ecosystem.”
But don’t expect the vendors to put everything together for you. It’s up to each individual company to assemble a collection of data management tools and technologies to fit their specific needs.
“Multi-cloud data management implementations typically are a collection of products, often from multiple vendors,” the Evaluator Group says. “It is advisable for IT departments to first inventory their existing technology assets, and from there make decisions regarding required additional required purchases. A range of vendors – and most notably data protection vendors – are making investments in these areas, bringing enhanced capabilities such as improved data migration and visibility to bear.”
The public cloud has transformed from a technological novelty to an established business platform in a relatively short period of time. As the value of data to a business’s bottom-line increases, it’s incumbent upon them to have the right tools and techniques for managing data in a safe and effective manner, no matter where that data resides.
This article originally appeared in sister publication Datanami.
