Being as I live in Portland, I thought I’d put out a few recommendations of places I like to eat in case you’re around since some people asked me.  Yes, I’m probably leaving off tons of stuff.  The food carts, the grilled cheese grill, but hey, I just wanted to put together a quick list.  Feel free to invite me.

Breakfast

Waffle Window – I’ll have ice cream on my waffles for breakfast.  Thanks

Quick Lunch

Por que No – Really good carne asada tacos.  Two locations.

Bunk Sandwiches – Super good.  No space to eat inside but great to grab a great Sandwich.

Kenny and Zuke’s – People love this place.  I think its pretty good.  Big sandwich.

Dinner or Big lunches

Asian Style

Bamboo Sushi – Awesome sushi and kobe beef hamburgers.  Love this place.  Get both.

Lucky Strike – Never been here, but here its fantastic.

Italian & Mediterianian Style

Acena – Never been here but here its amazing.

Serrato - Looks good.  Can’t remember if I ate here or not.  I think I did.

Apizza Scholls – Probably the best pizza in Portland or the world.

Western Asianish

Marrakesh – Belly dancers?  Eating on the floor?  Eat with your fingers?  Yes.

East India Co. – Even my Indian friends admit you can’t even get Indian food this good in India.

Portlandish

Paley’s – Want to know the name of the chicken you are eating?  Where it grew up?

Screen Door – Southern Cuisine.  Loved it.  Don’t remember much more than that.

Castagna – Northwest Cuisine.  Good hamburgers.  Also Pigs feet if you like that too.

Up North

If you are staying in Vancouver and don’t mind a quick trip east, check out LaPella.  Pretty good.  My wife and I ate there 2 weeks ago.

Looking forward to seeing everyone!

#!/bin/bash
IP=10.93.234.241/pictures
wget http://$IP/blade/B230.png
wget http://$IP/blade/B230.png
wget http://$IP/blade/B440.png
wget http://$IP/blade/Blade_full_width_front.png
wget http://$IP/blade/Blade_full_width_front.png
wget http://$IP/blade/Blade_half_width_front.png
wget http://$IP/blade/Blade_half_width_front.png
wget http://$IP/blade/Blade_half_width_front_marin.png
wget http://$IP/blade/Blade_half_width_front_marin.png
wget http://$IP/blade/SfBlade.png
wget http://$IP/blade/SfBlade.png
wget http://$IP/blade/sequoia_front.png
wget http://$IP/blade/sequoia_top.png
wget http://$IP/blade/silver_creek_front.png
wget http://$IP/blade/silver_creek_top.png
wget http://$IP/blade/ucs_b200_m3_front.png
wget http://$IP/blade/ucs_b200_m3_top.png
wget http://$IP/fi/switch_psu_DC.png
wget http://$IP/rack/Alameda_1_front.png
wget http://$IP/rack/Alameda_1_top.png
wget http://$IP/rack/Alameda_2_front.png
wget http://$IP/rack/Alameda_2_top.png
wget http://$IP/rack/Alpine_M2.png
wget http://$IP/rack/Alpine_M2_front.png
wget http://$IP/rack/C220M3_front_small.png
wget http://$IP/rack/C220M3_top.png
wget http://$IP/rack/C420_front.png
wget http://$IP/rack/C420_internal.png
wget http://$IP/rack/SD1_Gen2_front.png
wget http://$IP/rack/SD1_Gen2_front.png
wget http://$IP/rack/SD1_Gen2_internal.png
wget http://$IP/rack/SD1_Gen2_internal.png
wget http://$IP/rack/san_mateo_front.png
wget http://$IP/rack/san_mateo_internal.png
wget http://$IP/rack/sl2_front.png
wget http://$IP/rack/sl2_front.png
wget http://$IP/rack/sl2_top.png
wget http://$IP/rack/sl2_top.png
wget http://$IP/rack/st_louis_1u_front.png
wget http://$IP/rack/st_louis_1u_top.png
wget http://$IP/rack/st_louis_2u_front.png
wget http://$IP/rack/st_louis_2u_top.png

Bash is found on the MAC and Linux which comprises every computer in our house.  (we have a few windows VMs here and there but nothing we use).  And bash is fun cause you can get familiar with the command line.  We’ll start by using pico maybe?  And do a simple hello world program first:

Program #1:

#!/bin/bash
echo “Hello World!”

That’s pretty easy, but the next fun thing is to make it ask and answer a question:

Program #2:

#!/bin/bash

echo “What is your favorite color?”
read color
echo “${color} is a nice color”

 

Program #3:

#!/bin/bash

echo “What is your favorite color?”
read color
echo “${color} is a nice color”
if [ "$color" == "red" ]
then
echo ${color} is my favorite!
fi

Program #4

#!/bin/bash
echo “Can you guess my favorite color?”
while true
do
read color
if [ "$color" == "red" ]
then
echo “You guessed it! ${color} is my favorite color!”
break
else
echo “Nope. $color is not my favorite color. Guess again!”
fi
done

We’ll probably make it ask a few other questions and then some other cool things. Might be fun!

Much of my thinking has been confirmed and augmented by a Ted Talk I watched this week by Mitch Resnick.  In his talk, he affirms that just because people can code doesn’t mean we expect them to all be professional computer scientists or developers.  We don’t expect all people who learn how to write to become novelists or journalists.  Its just a basic skill that is needed in our day and age.

With his program “Scratch” that him and his team has made I think I’ve found the answer I was looking for.  I got home last night and downloaded it onto our family iMac.  It sits right in the kitchen and got my 9 year old and 6 year old started on it.  We started out with a picture of a “sprite”, or in our case, the default picture of a kitten.  We then created “controls” such as: “When I press the spacebar”.  Then underneath the control we did things like “change color” or move 10.  (the 10 is 10 pixels, but kids don’t really know that yet).  My kids would then keep pressing the space bar.  That’s when we introduced the “Forever” loop to them.  Amazing!  In just a quick 10 min, they understood loops and making things happen.

I’m hoping to do more with this and my kids.  I don’t want them to think of computer programming as dry and boring, but rather a creative medium for doing really cool things.  I am thankful for the people at MIT for making this possible.

The first thing I did was bought a SSD from Amazon.  I got the Samsung 840 Series Solid State drive.  The drive came 2 days later thanks to Amazon Prime.  Once it came it was like Christmas.  I opened up the box and threw everything away except the hard drive.

Next I attached it to my computer with a SATA wire.  I borrowed it from a great guy I know at Mountain States Networking named Justin Kurynny.  (He’s the last person in a long line of people that told me I *had* to get an SSD)

From there I just went and downloaded SuperDuper from the web.  It works with Mountain Lion too!  I’m still on Lion and it worked flawlessly.

I did this on a Friday night and went to bed while it copied.  It ended up taking 6 hours to copy 500GB.  I woke up the next morning ready to finish up.

First, I watched a video on how to take apart my MacBook Pro.  Seemed simple enough, and it was.

I layed everything out, took my old hard drive out and replaced it with the new SSD.

The only thing that got me was that the hard drive needed a special tool (or at least a tool I didn’t have) for the drive mount screws.  After a trip to Home Depot with  3 kids who made it take much longer than it should have I had my tool.  I came home and finished replacing it.

Once I got it on all was good.  Super easy.  I turned my MacBook on and it worked just like it did before but with a little more spring in its step.  (You know that same feeling you get when you walk through a shopping center and they’re playing Christmas music? …Ok, maybe not)

A great guy I know named Tony Bourke calls SSDs a “Charlton Heston” technology.  Because once you try it, the only way you’ll ever not use it again is if someone pries it from your cold dead hands.  Awesome.

This week I was very privileged to go to Salt Lake City to Fusion IO headquarters and get a deep dive on their technology and how it differentiates from other competitors in the high speed, low latency storage market.  (Which is really starting to just be the general storage market these days instead of something niche.)  It was super neat to go there with a bunch of my buddies from Cisco and I can’t thank them enough for having us and treating us so go.  This meeting was brought about primarily because Fusion IO and Cisco have introduced a new Mezzanine Fusion IO card for the B series blades.  Specifically:  The B200 M3 and B420 M3.  The card is the same as the ioDrive2  but in Cisco blade server form factor.  This first version is 758GB drive.  We had a great time and and learned a ton.

Fusion IO’s main product is branded as ioMemory.  Its marketed as a new memory tier.  The co-founder, David Flynn had the idea of taking Flash Memory,  putting it on a PCI card, slap a basic controller on it and putting it into the server.  The server would then see this as a hard drive.  By not using legacy protocols of SAS or SATA and using their own protocol with software, they were able to get IO latency down to microseconds from milliseconds.  Couple this with flash memory and it translates to more IOPS, which means applications that normally have to wait for disk reads and writes can do it on orders of magnitudes faster.  One of the examples they cited said that customers were getting 20 times the performance with these drives compared to using standard disk drive arrays.  (Not 20% better, but 20 x)  From that above linked Wikipedia article it shows that fastest 15k RPM hard drive will only get around 200 IOPS.  Compare that to a Fusion IO card that same article shows 140,000 IOPS.  (My notes also say that they are getting 500,000 IOPS, I’m not sure which is correct but the idea is that its blazing fast.)

If you aren’t familiar with the state of the data center, and I commented on this in myVMworld 2012 post, storage is one of the biggest problem.  Numerous blogs, articles, and talks show that storage is the biggest bottle neck and the largest expense in the data center.  Duncan Epping commented at VMworld on the topic of performance that “The network is usually blamed but storage is usually the problem”.   There is a famine happening in our data centers.  Applications are starved for data.  They are waiting and waiting to get their data from disks.  Applications today are like a bunch of hungry children at home whose moms went to the store to get food and are all stuck in traffic with other moms taking a long time to make that round trip.  Storage IO performance has not kept up with the spectacular rate that processing power has improved over the last decade or so.

What we have been doing for the last 10 years (me personally and others) is designing storage systems that we think will meet the performance requirements.  When we get up and running we soon find that the storage system doesn’t meet the performance needs so we throw more disks at them until it does.  Soon we have tons more capacity than we need and a bigger footprint.  I’m not alone in this.  This is standard practice.  Commenting on this, Jim Dawson, the Vice President of world wide sales wrote for all of us to see:  DFP = RIP.  This he said means:  Disks for performance is dead.  He also mentioned that his customers when he was at 3PAR were adding so many disks for performance that they asked him to make smaller disk sizes because they didn’t need capacity, they needed performance.

Flash Memory to the rescue

The reason you are probably hearing so much about flash memory now and not before is because the price of flash memory has fallen below the price of DRAM (the kind of memory that when you pull the power power it, it forgets everything that was in it).  Flash memory, specifically NAND flash, is the flash that’s used in Fusion IO, SSDs,  SSD arrays, and pretty much everything you see out there that’s called flash storage.   This type of memory when you pull the power doesn’t forget which bits were flipped to ones or zeros.  NAND flash are the building blocks for nearly all the fast storage you’ve been hearing about.  From  people making USB thumb drives, SSDs, PCI SAS, Violin, or Texas Memory Systems (now IBM) and make arrays with them using their own controllers, they’re all using NAND flash.

The difference is how the flash is accessed.  SSDs go through the SAS or SATA controllers that add significant over head.  That makes it slower since those are legacy protocols used for hard drive technology.  But if you have one in your mac book pro like I hope to have soon, then you are not complaining and its just fine.  Most of the Flash storage solutions out there are based on using SAS/SATA protocols to access flash storage: Nimble storage, whiptail, etc. Its more simple to develop because the protocols are already defined and they can concentrate on value add at the top, like putting more protocols or better management tools in it.

Fusion IO has two advantages over these technologies.  First, since they are on the PCI bus, they are closer to the processor so its much faster.  Second, they don’t have the overhead of a controller translating older protocols.  There’s a driver that sits on the OS that manages it all.  Since they don’t go through the standard protocols they can also add better monitoring tools and even add more on top of that to innovate cool solutions.  (ioTurbine is an example of this that I’ll get to in a minute)

Fusion IO secret sauce

The ioDrive2 card is main product.  Its a PCIe card with a bunch of 25nm NAND flash chips on it.  We had this amazing Fusion IO engineer named Bob Wood come in and talk to us about how it works.  He schooled us so hard I thought I was back incollege.  We were worried we were going to get more marketing but in the words of @ciscoservergeek: Our expectations were amazingly surpassed.

Flash memory has what’s called an Erase Block.  This is the smallest atomic unit that can be written.  As flash gets smaller having 3 or more electrons leave, or somehow get disturbed will cause the erase block to flip a bit and be wrong.  The controller software is then always looking to make sure things are still the way they should be.

A standard fusion IO card is built in with about 20% of spare capacity that is used for when erase blocks get contaminated or flipped too many times.  Bob equated it to standing on top of a mountain and being struck by lightning.  There’s only so many times you can be struck by lightning and still go on.  (Apparently NAND flash can handle it more than humans).  When one of these erase blocks is retired, the card draws from the 20% pool.  In addition, other erase blocks are reserved for features to handle more error checking.  More official information on this “Adaptive Flashback” is here.

I asked then:  So if I have a 750GB card, do I only get to see 600GB of space?  No, the 20% overhead plus other reserved pools is in addition to the 750GB, so you will see that much capacity.  I imagine that the raw capacity is probably from 900GB to 1TB.

Bob told us that the design of the card is the classic engineering tradeoff design and finding the ultimate efficiency.  You have to worry which NAND flash you use, multiple suppliers, price/ performance, how much you can fix in software, how much you need to make sure you are error checking vs speed, capacity vs. features, etc.  It sounded like a fun multivariable calculus problem.

The other thing that was cleared up to me was the nature of the product.  DoesFusion IO make one product thats a hard drive and the other one a memory cache?  No.  Physically, its one product.  But you can license software to give it more features.  You’ll hear messaging of ioTurbine and DirectCache from them.  Those marketing terms describe software functions you can put on top of the ioDrive2  by licensing software.  ioTurbine is for VMs and DirectCache is for bare metal.  Its essentially makes the card act as memory cache for the VM or physical machine.

And this is where I suspect Fusion IO will continue to innovate: Software on the NAND flash.  How to make it more useful and do more things.

Fusion IO Tradeoffs

Like every technology, there are tradeoffs and no single technology is going to solve all your data center needs.  Isn’t that why we pay architects so much money?  To gather all these great technologies and choose the best ones to meet the needs?  Anyway, here are some tradeoffs:

Price: Its no mystery that Fusion IO drives aren’t super cheap.  You can buy at least 2 very nice servers for the price of the card, but that may not solve your IO problem.  But if you look at it that you can instead buy Fusion IO rather than some supped up disk array, then it might actually be cheaper.  In fact, they showed a case studies where it over 70% cheaper than getting big storage arrays.

Redundancy and HA: If you have one card in the server, that’s a single point of failure.  Now granted there are no moving parts, so the MTBF goes up, but still you are putting lots of eggs in one basket.  If you have a modern application where redundancy is in the software then this isn’t going to be a problem for you.  For the legacy apps ran in most data centers Fusion IO talked to us about several different solutions you could use to do HA.  A lot of this sounded like what we used to do with xCAT to make it HA.  We’d use DRBD and Steeleye and those were the same things we were told about by Fusion IO.

Now there’s no reason you can’t buy two or more of these cards and put them in the same server and then just use software to RAID them together, but you’re not going to be able to do that in a B200 M3.  Further more, you’ll want to sync blocks between drives.  Fusion IO recognizes that people want this and that’s why ioN is a product that I think we’ll see lots more from.  (more on that in a second)

Capacity vs. Performance: 750GB drive is not too far away from the 1TB drives I can put in my servers.  Fusion IO told us about an Oracle survey where 56% of the big data clusters had less than 5TB of capacity.  That doesn’t sound like big data does it?  But big data isn’t really so much about size of the file as it is to gaining insight into lots of transactions and data points where each individual record can be quite small.  And in that game, performance is everything.  So even though you can’t get as much capacity on the Fusion IO drives, you can hopefully get the working set on there.  They showed examples where entire databases were run off the cards.  They also showed that in tiered storage designs the cards form yet another (or alternative?) tier by keeping most recently used data closer to the processor.

Shared Storage is still in vogue: Most of the customers I work with have a shared SAN that all the servers have access to.  Fusion IO cards are directly attached to individual servers.  Fusion IO addresses this with its ioN product which is essentially a shared block storage device created with standard servers and Fusion IO cards.  ioN then presents itself as an iSCSI or Fibre Channel Target.  It can be used in conjunction with a SAN as a storage accelerator.

The trends we have been hearing about lately show that distributed storage in commodity servers is the future.  Indeed, Gary one of the presenters mentioned that as well.  That would work very well for Fusion IO.  But this requires software.  Software Defined Storage.  (see what I did there?) Either something like Hadoop, Lustre, GPFS NSD could work on this today but probably not in the way people want for generic applications.  ioN right now only supports up to 3 servers.  (Sounds like VMware’s VSA doesn’t it?)  I think this technology shows great promise, but its not going to be able to replace the SAN in the data center right now.

TL;DR

Fusion IO is having tremendous success in the market place.  I like the Cisco andFusion IO partnership because it adds to Cisco’s storage portfolio partnerships and gives Cisco UCS users more options.

The thing that got me most excited was the ioN product.  By allowing the common man to build your own Violin memory / Texas Memory systems array with commodity servers, we’re getting more choices in how we do our storage.  It still has a bit to go before it can really replace a traditional storage array.  It doesn’t have snapshotting, de-duplication, and all those other cool features that your traditional storage has.  But just imagine:

- What if you could add SSDs and Spinning drives to your commodity servers along with the Fusion IO cards and ioN allowed you to use that as well?

- What if you could then had software that could do that auto tiering of putting most used data at the fastest Fusion IO cards?

- What if you added Atlantis iLIO for the deduplication in software to get that feature into ioN?

All of this points to one trend:  Software is the data center king.  Its got to have a good hardware design underneath, but when it comes down to it, Fusion IO is faster because its software is more efficient.

TL;DR on the TL;DR

Software defined everything is the king: but even the king needs a solid hardware architecture.

I just got back from SC’12 in Salt Lake City and it was as usual a fantastic event.  One of my favorite times of the year.  Most of this has to do with the fact that I’ve been going to this conference for almost 10 years so I’ve met a lot of great people.  Perhaps the saddest part about the conference was that I didn’t get to see nearly half of them.  I spent way too much time in the Hilton lounge on the 18th floor away from the maddening crowd doing my day job.  Oh well, it was still fun!

Cisco had a respectable booth this year.  Many people were surprised to see our rack mount servers and asked:  ”What is Cisco doing here?”.  A few years ago it made more sense because Cisco had acquired TopSpin and was a top InfiniBand vendor.  It wasn’t too much longer before Cisco shut down its InfiniBand business and went back to a converged network strategy for the data center that is based on Ethernet.  So why would it be back at SC’12? We do sell servers now, so that’s different.  But what would be compelling about Cisco UCS for an HPC solution?

It turns out a small team at Cisco including the great Jeff Squyres has been hard at work perfecting an ultra low latency solution for Ethernet.  Jeff has been involved in MPI longer than he’d care to admit (even from way back when HPC clusters solely used gigabit Ethernet).  He’s currently Cisco’s representative to the Open MPI project, which is what most people use today for HPC workloads.  MPI is a library which applications use to run across distributed systems.  I used to use these libraries a lot.  Usually what you do is when you compile an application like HPL, WRF, Pallas, etc. we would point it to the MPI libraries we wanted to use.  Then when the application runs it hooks into the APIs of those libraries and message passing happens.  Yay!

What Jeff and team found was that due to the virtualization capabilities of the Cisco VICs (Virtual Interface cards), they actually lend themselves quite well to offloading MPI tasks.  Before we get into that, lets make sure we understand the capabilities of the Cisco VIC.

Cisco VIC 1225

You can look at the specs here.  It looks like a dual port 10GbE card.  But what’s special about it is that its a CNA (converged network adapter) on steroids.  The original intent was for virtualization.  You can take this card and create 256 virtual interfaces.  If this card were plugged into a Cisco Nexus switch, then the network administrator could potentially see 256 interfaces that he could apply policies to.  What this means with VMware is that you can use hypervisor bypass (calledDirectPath I/O)  to get better performance and network visibility.  The Virtual Machine gets what it thinks is its own physical link.  There have been several versions of this card.  In rack mount servers, the P81E was the precursor.  In the blades, the M81KR is the old model while the VIC 1240 and the VIC 1280 are the current incarnations.

HPC on the VIC 1225

Now that you know what the VIC 1225 does in the VMware use case, you can think about it a different way.  Instead, think of all those VMs as just an application.  After all, that’s all an Operating System is.  Its just an application that runs other applications.  And when we run DirectPath I/O we’re basically just bypassing the local switch and giving the VM its own access to the I/O.  This can be perceived by the hypervisor that the particular lane is just running in user space.  Well, that’s exactly what we do with HPC.

The idea is that when an MPI application first launches and starts to use the VIC for message passing it has to start up following the blue arrows in the diagram above.  Once it hits the USNIC plugin to libibverbs it initiates its “boot strap” to set up the connection or to use one of the virtualized nics.  To do this it has to go to the kernel level and use the standard TCP stack that takes so much time and eliminates the latency.

Once this bootstrapped configuration is complete, the plugin then sets up a queue pair with the NIC and follows the red line, bypassing the kernel and OS stack.  This offloads CPU cycles and aids tremendously in cutting down the latency.

Jeff reports that they are able to get 1.7 microseconds when two of our C220 servers are connected back to back running a 19 byte packet.  That packet is an L2 frame: 1 byte payload and 18 bytes for the frame (6 Bytes for source MAC, 6 bytes for  destination MAC, 2 for Ethertype, 4 for CRC) .  1.7 microseconds back to back is pretty impressive.  To give a comparison, Infiniband back to back connection latencies can range between 500-600 nanoseconds to around 1.29 microseconds.

When you couple this card with the new Cisco Nexus 3548 switch which delivers about 190 nanoseconds port to port, then the latencies we have observed will now come out to around 2.3-2.4 microseconds.  YMMV.  Here’s how it breaks down:

- With back-to-back Lexingtons (VIC 1225), HRT pingpong latency is 1.7us
- With N3548 algo boost, add 190ns latency
- Prototype Open MPI USNIC support adds 300-400ns (Jeff thinks we can reduce this number over time)
So the MPI HRT pingpong latency over N3548 is somewhere around 2.3us

For many HPC applications that’s enough to compelling especially for using standard Ethernet

The final benefit is the management capability that you get with UCS Manager.  You can manage these rack mount environments the same way you manage blades.  You can give them service profiles and you keep the settings the same using policies and pooled resources.  Its very easy.  While UCS Manager works today with the rack mount servers, in order to use the USNIC capabilities with the ultra-low latency, you’ll have to wait until sometime in 2013.

The bottom line now is cost.  How much will it cost for Cisco gear compared to using standard rack mount servers with InfiniBand?   I’m willing to bet that Cisco in HPC is going to be price competitive, but the real savings are in operational costs:  You can get nearly the same benefits of InfiniBand without using a separate network.  That’s one less network to configure, manage, and maintain.  That’s one less vendor to have to deal with.

If you’re looking for a simple to manage HPC departmental cluster, this may just be the solution you’ve been waiting for.

Remember when storage was boring?  EMC, NetApp, blah blah blah.  Well those days are gone.  Its not so simple anymore.  The shear number of storage vendors on the showroom floor was a clear illustration that storage is still an open frontier like the wild west, or even like a new season of American Idol… (OK, maybe that last one was a bad analogy)

Sure EMC still leads in market share and NetApp is the fastest growing, but there is plenty of room for disruption.  There were several really good sessions on best practices for storage.  One of my favorite quotes came from Duncan Epping (of YellowBricks and Clustering Deep dive author fame):

“We always blame the network for our problems but its usually the storage that is at fault”

Indeed we see this in practice quite a lot.  People can’t buy more UCS because they’re constrained by their storage.  The network seems easy enough to blame but as server administrators are getting more comfortable with networking (mostly because they have to since the network access layer is now inside the server at the vSwitch) they’re starting to get that right much more often than the storage.

One of my favorite sessions was a Gartner Storage Best Practices session.  They had the following great messages:

“IOPS resolution is a multi dimensional problem that may not best addressed in the storage array”

This is why putting Fusion-io cards in a server may help with a tiered approach.  (BTW, these were announced to be inside UCS Blades now and should be available before the end of 2012).  It also explains why companies like Atlantis Computing with their ILIO product can offer big performance gains by offloading some of the work the storage array has to do as well as saving space.

This brings up another point from the Gartner session when talking about VDI (aka SHVD: server hosted virtual desktop)

“Storage costs present the #1 barrier to entry”

If you’re wondering when the year of the virtual desktop will come its when the organization has to buy new storage.  Over 40% of data center budget for new equipment goes into storage.  It pretty much pushes server and network equipment to the fringes.  That’s why I don’t ever think there will be a ‘year’ of the virtual desktop.  Instead, we’re in the beginning of the ‘decade of the virtual desktop’

The Tiered approach works as follows:  You have fast disks ( SSDs), slower disks (FC, SAS) and then the slowest disks:  SATA spinning at 7200 RPMS.  Ideally you want the data you use the most (the master copy of a windows VM) sitting on the fast storage, swap files on the mid tier, and lesser used workloads sitting down at the bottom.

The issue with the tiered approach (which NetApp doesn’t really have except for maybe the flash cache which is read only) is that you have to put the workloads where you think they will be.  And there’s a good chance you’ll get it wrong.

And that’s one reason there’s such a huge market out there for storage products that solve the issues of how to store and manage virtual machine files.  It used to be the OS sat on the server and the local disk was the best you could do.  Now with all of those VMs contending for IOPS, the storage is the bottleneck.  The new sexy bottleneck.  Sexy because there’s a lot of money to be made if you can convince people your solution is the best.

Using flash storage seems to be the sexy way to entice your customers.  SSD arrays were all the rage at VMworld.  Tiered solutions that allowed SSDs with FC/SAS/SATA were also quite popular.   I thought I’d go through some of the storage solutions I had a chance to visit with at VMworld.  Most of these are lesser known so it will be interesting to watch and see how the space changes in the next year:

Violin Memory

The Violin 6000 Series flash memory array can do 1 million IOPS with 4Gbps of bandwidth in a 3RU space.  The secret sauce is that they build their own flash memory controllers instead of using the standard SSDs that most flash array vendors use.  This probably isn’t the cheapest but its hard to beat in terms of speed.  This is the storage you buy when money is no object and you just want fast.  Just imagining this connected to some Nexus 5000s and a UCS full of B200 M3s makes me all tingling inside.

Whiptail

I didn’t see Whiptail on the floor but I have helped 2 UCS customers configure this and get it running.  Its cheaper and wicked fast, but offers little intelligence as to what is happening.  Just cram a bunch of SSD drives into the array and lets you go to town.  For some people, speed is all you need and they don’t care about fancy dashboards.

Tintri

I had a great conversation with a great guy at Tintri at the vFlipCup event on Monday night… Wish I could remember his name.  (BTW, I represented Team JBOD with @CiscoServerGeek and we apparently wrote checks that our team could not cache).   …But I digress… Tintri seemed to be a mix of Atlantis Computing and Whiptail.  Instead of having a VM do the block caching like ILIO does, you instead have that intelligence take place on the controller.  Couple that with an array of SSDs and life looks pretty good.  This seems better than Atlantis in that your chance of VM going down is greater than your chance of the storage array going down.

Pure Storage

Two of the bigger booths of storage companies that I hadn’t heard of were Pure Storage and Tegile.

Pure Storage is a flash array vendor that seemed to have the most sophisticated protocol support including iSCSI (BTW: for the record I really don’t like iSCSI.  When you have UCS, Fibre Channel is so easy and with the UCS Del Mar release later this year you have all the components you need for FCoE without buying any separate equipment)

Tegile

Tegile is a hybrid array that supports tiered storage.  The value is the deep integration with SSDs.  I would look at this offering as a less confusing EMC offering.  It offers SAN and NAS capabilities as well as data deduplication.  Pretty sweet system.

Summary

It seems like all the providers have some great niches and I think most people would be happy with any of these storage solutions.  I’d hate to leave this post without tipping my hat to EMC and NetApp who I work with and do a tremendous job.  There’s a reason they both have so many customers:  They build great products.  I should also call out Hitachi Storage.  Their own team admits they suck at marketing, but in terms of performance and reliability for mission critical apps, its hard to beat their rock solid solutions.  Its truly a company built and run by engineers.  That’s one reason their customers and me like them so much.

So if this post makes you feel all warm inside, that’s because storage is the new sexy and it is good for you to look at on company time.