At DockerCon EU there was a panel discussion on the future of micro services. There were some very good insights there that I wanted to capture as well as some things I took away from it:
Most of this is from Adrian Cockcroft. He has really emerged as a visionary in the open source DevOps world largely based on his work at Netflix. (Fun fact or name dropping moment: When Adrian worked at eBay and I was at IBM we worked together on a GPFS project, and he was the one who introduced me to LinkedIn)
Adrian and the panel basically said that all middleware or infrastructure management should be open source and only the SaaS based applications are things people will pay for. I wholeheartedly agree with this. There are two arguments that really stood out to make this point:
1. The days where a company can work in NDA mode for 12 months on a project and then come out with a product for infrastructure management is too slow. By that time an Open Source project has gone through 5 iterations and has more functionality.
2. People become suspicious of projects like middleware that require a huge investment of time to become knowledgeable on. Why would I want to gain skills on your proprietary middleware suite that may be inferior to something that is open source? Open Source skills are more transferrable than a proprietary system.
I think the next hotness to emerge (that is already emerging) is a way to manage micro services running in your data center, whether that be a private or public cloud. This will do more than just manage your docker containers. There will be networking functionality and application queuing schedulers. At present there seems to be two good solutions:
1. AWS, as they have already started with their Docker Schedule. This is still pretty remedial, but don’t count AWS out.
2. An open source project.
There will undoubtedly be proprietary solutions. I could see VMware deciding they need to manage docker containers in vSphere or something, and that could be nice for them. But I think these solutions will fail. The one that will emerge will be an open source tool championed by a community of very smart people.
Projects I see arising this year are things like Kubernetes, Panamax, Swarm, etc. These are all ways to manage containers and have a huge head start on any enterprise projects. I would love nothing better for a large company, like my employer, to spend time and develop tools like this for networking in an open source model. Something that is completely open, but backed by a big company that can support it.
One big epiphany I had while watching the panel is that micro services brings the infrastructure management closer to the application side of the house, furthering blur the lines between infrastructure and developers. We’ve talked about how the lines are blurring between storage, compute, and networking, but we should also understand that these lines are being blurred between development and infrastructure: This is what DevOps is.
A few more notes from Adrian’s “State of the Art in Microservices” talk. I’ve quoted them below, but I’m actually paraphrasing it then I have a few comments by it:
“Cloud is API driven infrastructure. Whether public or private, it needs to have APIs. Its not just about self service interfaces. It needs to have APIs.”
Too often in my own company we show the self-service interface of things like UCS Director or OpenStack but fail to show how it will help our infrastructure friends sell it internally to the application teams. APIs are one way we can do a better job of presenting it. The problem is that APIs just aren’t sexy unless you are a developer trying to get an environment. Then you want nothing other than that API so you don’t have to deal with the people that want to keep the environment up and static.
“DevOps is a Re-Org. Not just create a new team and buy a copy of Chef.”
From the book “The Phoenix Project” and in real life we see that a DevOps team is successful when it takes the brightest of each organization and charters them to work with each other.
“With micro services, we treat each internal team like they are an external team. We don’t need to know the details of the other micro services details, we only need to know the APIs and how we can communicate with them.”
“As we’ve moved towards Continuous integration: Cost and size of change has been reduced while the rate of change has increased. This can amount towards massive disruption.”
I have one colleague I know who is doing this at his company. This company he is in is largely static and does things the old way: 1 week to code, 2 days to build and so forth. Doing several builds a day has caused his team to really stand out and disrupt their current status quo. There are far reaching benefits to this and people have taken notice in a big way.
“IT transition: static machine, virtual machine, containers, AWS lambda. AWS lambda is firing up a container and running it to do one request.”
“Architectural diagrams with simple 3-tier architectures are dead. Architectural diagrams are now the ‘Death Star’ diagram. This means that there are a ton of micro services talking to each other.”
Adrian talked about Netflix, Guilt, and Hailo’s architecture. These were very good to see how enterprises are moving to this quick model.
This year in 2015 I expect most enterprises should have at least their development team use Docker intensely. I’m using it in my applications and I already see the benefits. I can’t wait to see what happens the rest of the year!
