Enterprise Kubernetes and container storage a work in progress
Containers and Kubernetes have entirely different storage and backup needs compared with traditional IT, industry experts say, requiring stakeholder input and new vendor offerings.
A decade after the introduction of the Kubernetes container platform, enterprise IT infrastructure teams are still conflicted on how to support the popular orchestrator compared with traditional IT needs.
Adopting Kubernetes for the enterprise has continued to vex infrastructure teams due to the lack of mature management tools and products available on the market. These tools for supporting enterprise demands should include persistent storage, backups and automation, according to storage industry analysts.
An IDC report last fall defined infrastructure supporting Kubernetes as "container data management," which is a separate set of needs for containers among infrastructure teams compared with DevOps or platform engineering demands. Other analysts believe that the specifics of the proposed definition are up for debate, but there is a need for enterprise tools to support development efforts without micromanagement or slowing development pipelines.
The duties of supporting container infrastructure in the enterprise fall on physical storage and VM specialists, said Johnny Yu, an analyst at IDC and co-author of the "Worldwide Container Data Management 2023 Vendor Assessment" report.
"There isn't even a standard industry term for what the IT operations teams are doing," Yu said. "[They] don't care about how Kubernetes actually works. [They] just know there's a new environment [they] need to set up for these applications to run."
Hierarchy of container needs
The IDC report defined container data management products as those that focus on containers exclusively, data management capabilities such as disaster recovery and migration capabilities, as well as container storage management for persistent data access.
Handling these needs shouldn't require storage administrators to become experts on Kubernetes or other container platforms, Yu said. Instead, a certain level of abstraction of Kubernetes storage or system demands into commonly used infrastructure terminology and software is sought among these customers.
DevOps, which focuses on software development, and platform engineering, which focuses on creating abstracted resources for developers, have different needs compared with IDC's proposed container data management. Container storage and container data protection aren't differentiated as they are in the traditional storage, according to the report, as "the container market is not yet at a point where this separation is necessary, nor are most customers actively seeking it."
Specific products that IDC highlighted as meeting its definition for container data management include Portworx by Pure Storage, NetApp's container storage offerings and SUSE's Longhorn. Others mentioned included Veritas, IBM, Veeam and Catalogic.
The earlier days of container adoption by developers caught many storage administrators off guard, and they now have very complex and interconnected systems of applications to support.
These early container applications might have used open source software such as Velero to dip into Kubernetes technology, Yu said, but these apps have escalated into mission-critical workloads requiring proper data management.
"When there's a new technology, there's an idea of implement and deploy first to take advantage of it, and ask questions later," Yu said. "As [enterprises are] stepping into [containers], they really need to plan ahead and see what their roadmap is for the next three to five years."
Container planning
The rise of containers -- particularly Kubernetes -- within the enterprise was left to run unchecked to the detriment of any overall plan or resource allocation, said Brent Ellis, an analyst at Forrester Research.
The people experimenting with Kubernetes five years ago had free rein. Centralized IT didn't want anything to do with it.
Brent EllisAnalyst, Forrester Research
Containers were seen as a complicated new technology and outside the domain of infrastructure, Ellis said.
"The people experimenting with Kubernetes five years ago had free rein," he said. "Centralized IT didn't want anything to do with it."
Dennis Smith, an analyst at Gartner, said the trepidation that enterprise IT felt about applications and mandatory storage uptime made sense, as supporting container applications wasn't a part of most IT departments' expertise.
"Years ago, I would not have typically recommended stateful container applications except for certain organizations who are very skilled," Smith said. "Containers have become much more ubiquitous, [but] many organizations are still not ready. They don't have the time or skills."
Neither Forrester's nor Gartner's research separated out infrastructure needs for container platforms compared with strictly container orchestration platforms. But both Smith and Ellis echoed Yu's contention that planning how to scale and adapt container workloads is the key to avoiding future logistical headaches.
Kubernetes' adoption of a standardized Container Storage Interface driver simplified the management for infrastructure teams, Ellis said. This standard allowed more storage or data backup vendors to connect their products and has proliferated among storage companies.
"I wouldn't say storage management is easy in Kubernetes, but it's much better than it was two to three years ago," he said.
As part of planning the infrastructure to support containers, IT should mull what workloads are best served from a container approach compared with traditional IT or VMs, Smith said. Vendors in the earlier days of containers, much like the cloud, promised a lift-and-shift approach, but customers ultimately found their applications underperforming while overpaying.
"There are vendors that have pushed lifting-and-shifting legacy workloads into containers," Smith said. "[Gartner was] very outspoken in what the risks were in an operation standpoint."
Many of these duties will likely shift under the control of platform engineers, who will likely have a firmer grasp of how developers use specific technologies and how services should be consumed. Infrastructure IT and storage admins could then focus on storage uptime and availability instead of specific pricing or rationing woes.
"The platform engineering movement is the new movement of IT centralization," Ellis said. "Platform engineers expose functionality."
Tim McCarthy is a news writer for TechTarget Editorial covering cloud and data storage.
