{"id":368,"date":"2011-04-29T23:01:20","date_gmt":"2011-04-30T05:01:20","guid":{"rendered":"http:\/\/benincosa.com\/blog\/?p=368"},"modified":"2014-11-19T11:25:16","modified_gmt":"2014-11-19T17:25:16","slug":"dell-ipmi-issues-with-xcat","status":"publish","type":"post","link":"https:\/\/benincosa.com\/?p=368","title":{"rendered":"Dell IPMI issues with xCAT"},"content":{"rendered":"<p>One of our customers had some Dell R410 machines that were humming along just nicely. \u00a0One day something happened and all of the sudden xCAT rpower stopped working. \u00a0Was it that we updated the firmware? \u00a0Was it new xCAT code? \u00a0We couldn&#8217;t figure it out.<\/p>\n<p>Our first epiphany came when we realized that ipmitool worked just fine with lan (IPMI 1.5) and lanplus (IPMI2.0). \u00a0ipmitool works? \u00a0Why didn&#8217;t xCAT? \u00a0It turns out that there was a problem authenticating. \u00a0In fact, if we used lanplus, we didn&#8217;t have to even enter the correct password and we could turn machines off and on!<\/p>\n<p># ipmitool -I lanplus -U user1 -P thispasswordisbogus -H node001-drac -C 0 mc info<\/p>\n<p>Wow! \u00a0That&#8217;s a major security violation. \u00a0We alerted Dell. \u00a0But that still didn&#8217;t get xCAT&#8217;s rpower working. \u00a0As a temp solution, we modified \/opt\/xcat\/lib\/perl\/xCAT\/IPMI.pm to not try IPMI 2.0 and instead use only IPMI 1.5. \u00a0This worked fine for some things, but the great necessities of rcons, reventlog, and rinv would have taken more time to get working&#8230; and after all this was a temporary patch right?<\/p>\n<p>So today I woke up determined to resolve the issue once and for all. \u00a0Working with Dell support (who were very nice and eager to help us) I figured out that there was a an IPMI Encryption key that was set to some random 40 character\u00a0hexadecimal\u00a0string. \u00a0How it was set, I still don&#8217;t know. \u00a0Viewing it in the iDRAC looked like this:<\/p>\n<div id='gallery-1' class='gallery galleryid-368 gallery-columns-2 gallery-size-thumbnail'><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon '>\n\t\t\t\t<a href='https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-1.tiff'><img width=\"1\" height=\"1\" src=\"https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-1.tiff\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-2.png'><img width=\"150\" height=\"54\" src=\"https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-2.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/a>\n\t\t\t<\/div><\/figure><figure class='gallery-item'>\n\t\t\t<div class='gallery-icon landscape'>\n\t\t\t\t<a href='https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-21.png'><img width=\"150\" height=\"54\" src=\"https:\/\/benincosa.com\/wp-content\/uploads\/2011\/04\/PastedGraphic-21.png\" class=\"attachment-thumbnail size-thumbnail\" alt=\"\" decoding=\"async\" loading=\"lazy\" \/><\/a>\n\t\t\t<\/div><\/figure>\n\t\t<\/div>\n\n<p>We instead cleared that key and set it to &#8217;00&#8217; \u00a0(It was required to be an even number of hexadecimal characters). \u00a0Doing this solved out IPMI issue. \u00a0xCAT rpower then worked without a problem. \u00a0Dell then gave us a way to run this via the command line:<\/p>\n<p>racadm \u00a0-r drac-comp036 -u user1 -p asdfasdf config -g cfgIpmiLan -o cfgIpmiEncryptionKey 0000000000000000000000000000000000000000000000<\/p>\n<p>With that, order was restored.<\/p>\n<p>This also shows one great thing about xCAT: \u00a0The IPMI packets coming back were not authenticating correctly. \u00a0There was a problem with the way the challenges were coming back. \u00a0ipmitool seemed to be very forgiving about that and not care. \u00a0xCAT didn&#8217;t like it at all and would not let it pass. \u00a0We view this as an ipmitool bug than an xCAT bug. \u00a0Wouldn&#8217;t you rather know about a potential security problem?<\/p>\n","protected":false},"excerpt":{"rendered":"<p>One of our customers had some Dell R410 machines that were humming along just nicely. \u00a0One day something happened and all of the sudden xCAT rpower stopped working. \u00a0Was it that we updated the firmware? \u00a0Was it new xCAT code? \u00a0We couldn&#8217;t figure it out. Our first epiphany came when we realized that ipmitool worked&#8230;<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[116,916],"tags":[],"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/368"}],"collection":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=368"}],"version-history":[{"count":3,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/368\/revisions"}],"predecessor-version":[{"id":2794,"href":"https:\/\/benincosa.com\/index.php?rest_route=\/wp\/v2\/posts\/368\/revisions\/2794"}],"wp:attachment":[{"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=368"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=368"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/benincosa.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=368"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}