The CCIE Data Center Certification Process

CCIEData_Center_UseLogo

 

On July 9th, 2014 I passed the CCIE Data Center lab exam in San Jose earning me the CCIE certification.  Hurray!  When my team heard that I had done it, their response was:  If Vallard can do it, so can I!  Ha ha.  So needless to say a few more people have started down the path to certification, of which I have no doubt they will surely reach.

I have to say it feels pretty great and the process I went through to get it was very rewarding in that it deepened my understanding of data center architectures as well as the solid hands on skills required to implement these solutions.  With the CCIE certification, its the journey that makes it so worth it.

I thought I would write a bit of my experience of the process and how I approached it.  To summarize, it took me 5 times until I passed the written exam and once I did that I passed the lab exam on my second try.  I’m not saying my approach is the best, but it worked for me and I’m happy with the outcome.  The funny thing is, even though I worked really hard and learned so much to get it, I still feel like there are many things I don’t know about the platforms.  One of the drawbacks of my position is I don’t do a lot of troubleshooting with my customers because most of the solutions Cisco offers work really well.  Take UCS for example:  I spend probably 2 hours a month at the most troubleshooting issues with it – And that’s with the hundreds of UCS systems that my customers have that I support!

In spite of that, I still know this stuff very well now.  When a coworker asked me how to configure VPCs on the Nexus 5548s – to just give him a quick and dirty config – I was able to spit it all out from memory and I knew it was right.  I’ve done it so many times now I can do it in my sleep.

Need an OTV config?  I got you covered there too.  I can do OTV on a stick light speed setting it up with multicast or adjacency servers, I don’t even care.  I can do it all.  Boom.  So yes, passing the CCIE exam gives you confidence because you learn a ton.  That’s kind of how I felt when I graduated with my computer science degree from Berkeley.  Even though the program kicked my trash and made me feel like a sorry sucker most the time, it made me believe that armed with the skills I could do anything… given enough time.

So here’s my experience:

The Written Exam

The CCIE Data Center written exam topics are spelled out pretty clear on the Cisco Learning Network page.    I first took the test, in its beta form and I knew very little about the Nexus product line other than a few switches I had set up before.  I took the test without studying.  Zero prep.  Didn’t even look to see what was on it.  You see, I had to have humility beaten into me.  Anyway, I failed miserably.  Seriously.  I thought I was the man at UCS.  I got less than 20% right on it.  I blamed it on the way  the questions were worded, but in hind site, there were very clear answers that stood out among the wrong ones.  The thing was, it was hard.

After my first failure in August 2012, I gave up for about a year, not thinking it was for me.  Then I learned that a few more friends had already passed the written and were working towards the lab.  My pride made me think the same thing my team mates thought when I passed:  “If they can do it, then I can do it.”  My method, I thought would be a brute force attack on the exam.   So I took the exam again a year later in July 2013 after really working specifically on Nexus and MDS.  I felt that if given the beta exam again I could pass it.  The problem was, the exam was much different than I remembered it and again I did poorly.  When I failed, I rescheduled after realizing a few of my mistakes.  I took it again in August and September each time doing a little better, but each time not quite getting it.  By the time my December test came I was solidly prepared and just before Christmas on the 23rd of December I passed the written.

So what’s my advice on the written:

1.  If you already work in this field and have hands on with Nexus, MDS, & UCS, take the exam to see what’s on it.  CCIE is a total investment and if it takes you a few hundred dollars to pass the written exam, it might be worth it.

2.  If you fail the first time, take it again as soon as you can.  I think there are new rules going into affect that make it so you might not be able to take it as often.  However, once you start down the road to CCIE certification, you can’t stop until you’ve reached the end.  Otherwise you lose it.  That year I spent off was a waste.  I should have kept going.

3.  Once you pass the written exam, schedule the lab exam as soon as possible.  There are several months of waiting time right now and you don’t want this train to stop, so keep working towards it.

The CCIE Data Center Lab Exam

My entire IT career has been spent doing very hands on things.  I’m fortunate in that when I learn how to do something via the command line, my fingers seem to remember how to do it pretty well.  In some ways that’s bad because I have a hard time explaining things (which means maybe I don’t know how it works in the first place?) But I can usually always get things to work.  Being a fast typer helps as well.

As soon as I passed my written exam, I scheduled the lab.  The soonest I could get in was April 15, 2014.  That’s right: 4 months out. I had very little to go by other than the blueprint and Bryan McGahan’s excellent writeup.  I flew in the night before, and went to bed around 10PM, but then at 3AM had trouble going back to sleep.  I tossed and turned until about 5AM and then finally just got up, went for a 3 mile run, ate a good breakfast and showed up at Building C in San Jose 30 minutes early.  I sat in the waiting room with 14 other nervous people.  Man, I was tense.  I hadn’t felt that way since finals in undergrad.  We finally went in and I went to work.

As I was taking the exam, I tried to get that zen experience that Brian talked about in his blog, but it didn’t happen for me at all.  In fact, hardly anything happened for me.  For some reason, though, I thought I had done pretty well.  Wrong.  0% in multiple categories.

But I didn’t go into this thing the first time blindly.  I How did I prepare?  Hands on baby.  Stick time.  Yeah!

I was fortunate enough to have a pretty decent lab.  My equipment was good, but not complete.  I had:

– UCS with the old 6120s (but I’ve worked on plenty of 6248s so I wasn’t worried if that’s what they would have in the lab since I know all about unified ports.).  But 6120 fabric interconnects was all that was available to me.

– One Nexus 7010.  I had 1 Sup1 but I upgraded it to 8GB of RAM so that I could do 4+1 VDCs.  Didn’t matter, 4 would have been fine, since what I really needed was 8 VDCs.  But I made due.  I had one M1 line card and one F1 line card so that I could practice OTV, LISP, FabricPath, FCoE and layer 3 stuff.

– One Nexus 5548.  No line modules but I was fortunate enough to have layer 3 capabilities.  This helped me when I practiced OTV.  I also had several Nexus 2148s hanging around so I could do FEX things, but I could only do so much with a single Nexus 5548.

– One MDS 9148 Fibre Channel switch.  He worked pretty well.

I had a great base to get going on but in the end, I just couldn’t put it all together.  Why did I fail the first time?  Two reasons I think:

1.  Lack of confidence.  This is a big deal.  Nobody expected me to pass.  I’ve only been at Cisco for 3 years and I know people  who have been here a long time and haven’t earned the CCIE certification.  The second time I went in, I told my manager that I was getting it.  I was solidly prepared.

2.  Lack of equipment.  This was the biggest reason in my mind.  I’m cocky (conceited? immature? ignorant?) enough to think I can do these things.  I have young 4 children, and I’ve watched them all alone for 4 days straight, so I’ve already faced huge challenges!  I can do this!  If you look at the lab information and the equipment they use, you can see that I’m somewhat lacking.  For example, I had no director class fibre channel switch and not enough equipment to fully test things out.  This is one of the biggest barriers to passing the CCIE data center exam:  Having the equipment.  You are at least looking at several million dollars here and that’s probably why renting is such a good option and makes a ton of sense!

Anyway, here are my tips for the lab, when I passed, as well as for life in general:

Tip 1:  Higher is lower/ lower is higher?

I was also informed of a very cool trick.  When you think about priorities of different protocols or features, there’s an easy way to remember it.  This was taught to me by Ryan Boyd, a great guy I work with:  If its a layer 2 protocol (LACP, fibre channel stuff, VCP, spanning tree) the lower the number means higher the priority.  If its a layer 3 protocol (OSPF, EIGRP, OTV, VRRP, etc) higher the number higher the priority.  Fabric path is tricky, because its supposedly layer 2, but when you realize that its running IS-IS as the control plane then it makes more sense that it falls under the layer 3 rule: The higher the number, the higher the priority.  Why didn’t anyone tell me this before?

Tip 2:  copy & paste

I had several people tell me they use notepad, copy the command line stuff into it and then just put it in.  One of my friends told me he did that and blew away his switch and had to start from scratch.  This takes away far too many precious minutes from your lab time.  Lab day is one of the fastest days ever.  I spent a lot of time trying to debug something in the lab the day I passed.  When I looked at the clock, I realized that I had just spent 45 minutes burning away lab time.  Bad form!  (Fortunately, I had everything else done) So I don’t copy and paste.  I just type it out on the command line.  I have really good typing skills.  Its the one thing in high school that I did on a typewriter that really helped and has stuck with me.  Plus, writing all that code in college got me pretty good as well.  So for me it was type away.  Even if I’m doing the same thing on multiple switches.

As an aside:  The other funny thing I noticed is that people that do Cisco switches don’t type in all the words.  They do things like

sh run or sh int brie

Since I have big Linux roots, I do a lot of tabbing.  So maybe I add one extra keystroke, but this works for me.

Tip 3:

Draw it out.  In Brian’s blog he shows how he spent the first hour drawing it out.  I didn’t do quite that much the second time when I passed, but I did read through each section before I started working on that section.  This helped me when I had to remember which interfaces were connected to where.  You get as much scratch paper as you want.  I used more than the average.

After I failed the first test, I scheduled the second lab attempt as soon as I could.  The problem was:  The next available time was in September!!  Wow.  So I checked every day, several times a day for an opening.  After 3 days of this, I got July 9th.  So my lesson of not getting off the train helped out.  I thought:  Let’s keep going.

My friends had recommended INE labs and those things are *really* good.  I read through some of my friends labs, but didn’t use any of them.  Instead, a colleague of mine was building a lab out of spare parts and I joined forces and we built it together.  I like this approach a lot because I like touching hardware.  I like knowing how to set it up from scratch.  I’ve always done this.  We got a study group together of people that were going to take the lab exam and we hammered through all kinds of scenarios, really making sure we knew how to do it.  I’ll never forget watching the USA play in the world cup trying to get all our components working.

I tore the lab up several times and the week before the test, I really went to town.  (UCS, N1kv, MDS, N7k, N5k on the 4th of July is super patriotic, so that’s how I celebrated!)  I was continuing to go through it all the way up until 11PM the night before the test.  By that point, I had had enough.  I felt super ready.  I slept all the way until 6AM, extremely thankful I didn’t wake up at 3AM again.  I was still really nervous.  I got to building C early.

10518018_830156400328249_982627123_nThis time I had experience and I blew through all the questions keeping track of points feeling like I got nearly everything.  By lunch time I felt really good.  By 2PM I was sure I was passing… if only I could get this one thing working… I got it working by 3PM by being calm and retracing my steps.  I spent the remaining time going through the questions and making sure I had answered them right, tweaking things here and there and finding some things I had forgot. I counted the points and even though there were some things I never got working, I felt pretty sure I had enough to make it happen.

I left San Jose and went to the airport.  I called my wife and told her I felt good, but still wasn’t sure.  What if I missed something?  What if I didn’t save something?  (But I remember saving at least 3 times on every item before I left, so I was pretty sure about that)  Before I boarded the plan an email came.  I opened it up.  Put my hands in the air and jumped for joy.  The people in the airport probably thought I had just won the lottery.  But this wasn’t luck my friends, this was being prepared.  I had passed.  I texted my manager a few good friends and thanked them for their support.  It was a good day.