In xCAT we’ve been doing ram-root solutions since 2005.\u00a0 We call it ‘stateless’.\u00a0 Since 2005 there have been a lot of other ‘stateless’ solutions that don’t necessarily match our definition.\u00a0 Fair enough.\u00a0 You can call it what you like.\u00a0\u00a0 Red Hat for example calls ‘stateless’ an NFS root solution.<\/p>\n
We have ignored NFS root for a while because ram-root seemed to be the best for HPC applications.\u00a0 The tradeoffs between NFS root and RAM root are the following:<\/p>\n
NFS root = more network traffic<\/p>\n
RAM root = less ram to use on your system.<\/p>\n
RAM root has been very good to us because these days with Nehalem processors we normally see servers with 24GB of RAM.\u00a0 So to use 300MB at the most (with a bloated InfiniBand stack in it) doesn’t seem that bad.<\/p>\n
However, if we consider a hypervisor running a slew of virtual machines, having a bunch of copies of the same thing in memory doesn’t make that much sense.\u00a0 Especially if you consider that if we do an NFS root then the files will be read only.\u00a0 NFS root also allows us to lock down the machine in a way that RAM root doesn’t.\u00a0 But that can create some complexity:\u00a0 NFS root you need to tell which files are read only and which ones can be writable.<\/p>\n
xCAT will soon have an NFS root solution.\u00a0 We are examining other implementations to see what we can ‘steal’.<\/p>\n
What I have learned is that you can’t just give any standard kernel the nfsroot=<path option>.\u00a0 This is because the proper modules are not all in the kernel to do NFS mounts.\u00a0 So the kernel dies.\u00a0 So we have to give it a ram disk.\u00a0 The secret sauce of xCAT will be in the ramdisk start up file.\u00a0 This is where we can scale, do random waits, and mount everything nicely.\u00a0 We also should put an xCAT client in the initial ramdisk so that we can tell the server where we are.\u00a0 Once you have it all up and going in the ramdisk, the last step is:<\/p>\n
exec switch_root -c \/dev\/console \/sysroot \/sbin\/init<\/p>\n
The magic is in between that last step and the kernel loading the initrd.<\/p>\n","protected":false},"excerpt":{"rendered":"
In xCAT we’ve been doing ram-root solutions since 2005.\u00a0 We call it ‘stateless’.\u00a0 Since 2005 there have been a lot of other ‘stateless’ solutions that don’t necessarily match our definition.\u00a0 Fair enough.\u00a0 You can call it what you like.\u00a0\u00a0 Red Hat for example calls ‘stateless’ an NFS root solution. We have ignored NFS root for…<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[1],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/68"}],"collection":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=68"}],"version-history":[{"count":3,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions"}],"predecessor-version":[{"id":2844,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/68\/revisions\/2844"}],"wp:attachment":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=68"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=68"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=68"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}