Lately I came across a few bare metal servers in our lab and decided I would test some of my ideas on setting up a bare metal kubernetes cluster on these nodes. I started thinking that if I were to do so I would need some persistent storage. There were a few options I could choose from:
- I could NFS mount to all the nodes a server that had a ton of disks in it. I could also find an old NetApp array or something else in here that may do the trick.
- I could use Cisco Hyperflex. The only problem was that HX requires VMware or Windows, both are not options for me as I refuse to pay their licensing costs.
- One project minio caught my eye as a great solution as I could get great object store on prem. It is already managed by kubernetes so it becomes native storage. I loved this idea.
As I started thinking more about this I realized how great the Kubernetes Storage Interface idea is. Looking at Hyperflex, a lot of the heavy lifting done in the code is managing the nodes and installing the controllers. Much of this Kubernetes gives you for free. The future then of distributed storage that we’ve been getting from VMware with VSAN or HX, or Nutanix is looking more to me of a place that Kubernetes can dominate.
Another project that holds great promise to me is Rook. Rook uses the idea of Kubernetes Operators to set up distributed storage including Minio and Ceph. Operators become the expert in deploying and running these systems.
Examine what containers have done for software development: A new way to easily distribute and run applications in consistent environments. Now consider, by extension that Kubernetes is using some of that goodness to deploy infrastructure. It used to be pretty complicated to setup and manage the lifecycle of databases, Kafka streams, or other services required by the applications. Now, though, Kubernetes is making this easier for us.
I’ll be examining some of these ideas in some upcoming posts, but I’m pretty excited about what these new features enable us.
Finally, I’d say: If you are a storage vendor or selling storage software, if you don’t have a kubernetes strategy, its time to start looking at it. Not only will it increase your time to market, but it will make the upgrading, lifecycle, and maintenance of your storage system for free. And I’m not talking about how Kubernetes can run on top of your storage, I’m talking about how your storage should be running inside Kubernetes.
Update: Check out Rook / Ceph in action:
