The xCAT 2010 Year in Review

DISCLAIMER:  I’m not the official gatekeeper of xCAT, and I don’t work for IBM. But as a large contributor to the xCAT project, as well as a user, I figured I had just as much authorization to write about what happened to xCAT in 2010 as anyone else does.  And the best part is:  I’m not tied to a large corporation that monitors what I can or can not say.  So I can say, and will say anything I want to :-).  So anyway, what I say here represents my opinions and views and don’t represent the opinions or views of the priests at IBM nor of the greater xCAT community.  So tell your lawyers to go away.

So If nothing else, this is to give you something to read about while you’re recovering from your new years eve party.  I present:  The xCAT 2010 Year in review:

To start off with, I just want to thank all the users and developers that I’ve been able to work with this year. It’s been nothing short of incredibly amazing.  We’ve been working on tough problems.  The xCAT user base has some of the most talented, passionate, and dedicated people in the industry.  (And in most cases are smarter than us developers)  It was great to work with all of them this year.  I am thankful for your insites and criticism.  We hope people become more critical of xCAT and don’t sugar coat anything.  I didn’t work with one person this year on xCAT problems that I didn’t walk away from thinking: “Hmm, that <dude|lady> is pretty smart”.  So the talent inside the xCAT users group is fantastic.  You can tell that by the types of comments that come into the mailing list.  They’re much different than what you might see on other open source projects.  So the biggest story of 2010 for xCAT:  The users moved mountains, and helped make xCAT better.

Enough flattery.  As far as releases, xCAT went from the 2.3.4 release in March and culminated with 2.5.1 that was released Dec 11th.  Its nice to see the releases coming so rapidly.  I don’t anticipate this will be the case in 2011.  I think we’ll see slower major release cycles and minor release cycles will just coincide with OS updates (VMware’s updates, RHEL6 stuff, SLES, etc.)  Most of the reason I think things may slow down is because xCAT is pretty feature complete.  It can do lots of things.  It could use lots of clean up work, but nobody seems to be interested in doing that.  Most developers I think just accept that no error messages are printed when XYZ happens, and look at it as a learning opportunity for you to become acquainted with the xCAT code.  That’s too bad in my opinion.

Technology Highlights

There lots of features added to xCAT that made its awesomeness ooze even more than ever.  As usual, I usually skip all the awesomeness that was added to xCAT’s IBM’s AIX and SystemP functionality, since I don’t ever work on that aspect of xCAT, and usually just complain about the table pollution the AIX tables cause. (You know who you are: ppc, ppcdirect, pchcp, nimimage, etc) However, from what little I do know they’ve been pretty busy with it this year and I am glad to work with them on some small scale on this great product.

VMware Support – VMware ESX support has been in xCAT for a long time.  But this year we added better stateless ESXi support (The first ever) and also ESXi kickstart support with ESXi 4.1 (I think we’re the only project that has that).  We also added cloning, thin or thick, and lots of other cool features to support VMware virtual machines.

On the KVM front, we did the same if not better.  So if you’re running VMware or KVM on your nodes, xCAT doesn’t care, it looks the same.

imgimport/imgexport – We started the ability to import or export images (stateless and stateful) into xCAT.   This is still not as mainstream as I’d like it to be, but we hope to add more to this in 2011.  I still dream of supporting some type of physical machine image store.

Dynamic DNS – Dynamic DNS was added to xCAT and I don’t think used as often.  But it assigns discovered nodes static IP addresses so that  you don’t need to predefine them in /etc/hosts.  This is great and a step further to making automagic discovery even more magical.

Statelite – xCAT has had stateless (RAM root in tmpfs ) support since 2005.  It’s also had a type of hybrid with NFS.  But now we’ve made statelite:  This allows stateless, with NFS root, with a real hard drive for statefull files.  Its wild.  Give it a try.  It probably offers the most extreme way to manage a system.  The coolness of the hierarchical tree support is frightening.

Sumavisor – This isn’t xCAT proper, but a nice Web Interface to xCAT has finally been added by Sumavi.  (More on that in the next section)

Non Technological Highlights

I think a big highlight is the emergence of a company that is dedicated to bringing xCAT to the masses and is not afraid to invest in it.  This is my company, Sumavi , founded Feb 2010 :-).  We have done very well this year with service engagements with some big time accounts.  In addition we’ve made some great partners, turned out some solid documentation and made a really nice GUI front end to xCAT that we’ve branded ‘The Sumavisor’.  And its not just a GUI.  It does much more than that, enhancing xCAT to make it look polished and add control and insight that you can’t get on the command line.  It also does a lot to cut the learning curve down and adds commercial support to all xCAT installations regardless of what hardware they are running on.

As far as the community goes, I had a rant on xCAT’s documentation problem.  Others at IBM have attempted to make the xCAT documentation more usable.  You can see it here.  It’s definitely better than it was and is a step in the right direction, but I feel like the Sumavi documentation is much more usable.  But you tell me.

In other news, we rolled out xCAT in banks, credit card companies, and the usual blend of government and university accounts.  But the most exciting is to see xCAT venture into corporations that are not focused on HPC.  This year we installed the Lego Universe MMOG environment using xCAT.  It was an all Windows environment too!  We did some cool stuff all over the place.  Even while not at IBM I was able to talk to developers at IBM and from all over the world.

In addition, we really broke out and started supporting hardware from Dell, HP, and other whiteboxes.  I even started developing the Cisco UCS xCAT module.  I haven’t finished it yet… I’ll wait for Cisco to cough up some support dollars for that.  You hearing me Cisco?

Predictions for 2011

2011 I hope we will continue to see xCAT do more outside of HPC. I hope to get into more cloud deployments.  We’ve already done a bunch.  But we’d like to have more packaged products.  I see us coming out with EC2 support. (not based on anything but Amazon APIs (sorry, but we heard too many complaints about the open source version of Eucalyptus)), I also see more appliance based models, like Hadoops.

As more people want to drive xCAT, ad Web Services API is in the works.  Right now you can perform xCAT calls via XML messages to port 3001 with xCAT, but this needs to be more robust.  We’ve done that with the Sumavisor, but there is more that needs to be done.  Hopefully that will be out in the first half of the year.

And finally, the biggest thing about cloud is that its all about the applications.  How will we deal with making applications more agile?  I see this as a major focus for our group.  Creating virtual machines, etc is great, but how do we help, or is it even our role to help with the creation of the contents of those machines?  We seem to be in that world already.  But where or should we draw the line between xCAT and things like RightScale.  Where or should we draw the line between xCAT and Chef, Puppet, etc?

I don’t know yet and I can’t wait to find out.

Anyway, I hope you had a wonderful 2010 and I hope 2011 is just as wild for you!

  • Pidad

    Hi, Any thoughts on how does xCAT and Chef stand separate as of today? What are the key differentiating factors? Any pointers in this regard would be helpful.nn-Pidad

    • Anonymous

      Its an interesting topic to think about. They’ve both wondered a bit into each others space. But fundamentally, they run by two different philosophies.nnxCAT was created as a bare metal provisioner. It was a day 0 get the cluster up and running. As we moved to stateless we started saying: Well, if you want to change anything, just change the image. And so we really just started doing images. But before that we had post scripts. Egan wrote a rules engine that was a simple starter to what Chef or Puppet are today. We didn’t really pursue it though and used it mostly for installation. Again the thought was: If you need to change something, you should just reinstall it to make sure everything can be reproducible during installation.nnChef, puppet, and all those other tools that are in vogue today came from a more enterprise angle: The system is already stood up, we don’t want to reinstall it, just update things. And so they didn’t use it as a post install mechanism like xCAT does, but instead an ongoing maintenance tool. And they do it very well. As those tools have advanced they’ve moved down the stack to bare metal provisioning.nnFrom a feature standpoint, xCAT provisions much better and has far more capabilities (stateless, statelite). Chef and Puppet do configuration management much better. As xCAT has advanced its gone to provisioning different things now. VMs, Storage, and to basically be a data center operating system. Chef can and has been used in the past as the configuration management that is done after xCAT installs the hardware. nThat’s my take anyway.

  • Pingback: Sam Sean

  • Pingback: michael kors cheap