xCAT r* commands with UCS

xCAT out of the box works on UCS.  Or UCS out of the box works with xCAT? Whichever way you look at it, it works. All of the cool things you can do with xCAT like provision nodes, KVM, vSphere, stateless computing, etc, can all be done with UCS.  In fact, you can even run most of the r* commands on UCS.

Cisco UCS allows this through IPMI.  And configuring IPMI on UCS is easier than any other system I’ve ever used.  While I still plan on furthering my xCAT UCS plugin to get more capabilities into xCAT, most xCAT functions can be used with UCS managing the servers with IPMI.  For most people, this is good enough.

Using IPMI this is what seems to work with xCAT 2.6.6 and UCSM 2.0(1): (See the end of this for sample output)

  • rpower on|off|stat|boot
  • rbeacon on|off
  • reventlog [clear]
  • rvitals  (this is quite thorough)

rinv seems to hang on me.  This I think is due to the nature of service profiles, where UUIDs and MAC addresses are transient.  I’ll investigate this further.

So how do you do it?

Configuring an IPMI machine with xCAT has been well documented.  What I haven’t seen documented so much is configuring IPMI inside UCS.  This is surprisingly easy.  Here’s how its done:

1.  Create a Service Profile Template that you will apply to your blades.  This is documented very well in various places so I won’t go into it here.  Creating a Service Profile Template is UCS 101.   After you’ve created your service profile, assuming its an updating template you can proceed to the next step.  (Don’t worry, any changes made for doing IPMI don’t require a reboot)

2.  From the Servers tab, filter by Service Profile Templates, and navigate to your service profile template.

3.  Click on the policies table and look at the IPMI Access Profile Policy

4.  Create a new policy.  In this policy you’ll give the name of the user and give it a password.  Make sure they have admin privileges.  For simplicity, I just made my user and password the same as my UCSM user and password.

5.  Apply the setting and click save.

From here on out you can just run IPMI commands.  The only issue now is to know which IP address corresponds to the IPMI interface of which blade.

This can be found in UCSM under the Admin tab, Communication Management, Management IP pool.  If you click on the IP addresses tab on the left hand side, you’ll see all the IP addresses.  

Ok my friend, you now have it. xCAT running rpower commands.

And now, here is a sample output running rvitals on a UCS B200 M1:

# rvitals lucky01
lucky01: BIOSPOST_TIMEOUT: N/A
lucky01: BIOS_POST_CMPLT: 0
lucky01: CATERR_N: 0
lucky01: CPUS_PRCHT_N: 0
lucky01: DDR3_P1_A1_ECC: 0 error
lucky01: DDR3_P1_A1_PRS: 0
lucky01: DDR3_P1_A1_TMP: 26 C (79 F)
lucky01: DDR3_P1_A2_ECC: 0 error
lucky01: DDR3_P1_A2_PRS: 0
lucky01: DDR3_P1_A2_TMP: 25 C (77 F)
lucky01: DDR3_P1_B1_ECC: 0 error
lucky01: DDR3_P1_B1_PRS: 0
lucky01: DDR3_P1_B1_TMP: 26 C (79 F)
lucky01: DDR3_P1_B2_ECC: 0 error
lucky01: DDR3_P1_B2_PRS: 0
lucky01: DDR3_P1_B2_TMP: 27 C (81 F)
lucky01: DDR3_P1_C1_ECC: 0 error
lucky01: DDR3_P1_C1_PRS: 0
lucky01: DDR3_P1_C1_TMP: 24 C (75 F)
lucky01: DDR3_P1_C2_ECC: 0 error
lucky01: DDR3_P1_C2_PRS: 0
lucky01: DDR3_P1_C2_TMP: 25 C (77 F)
lucky01: DDR3_P2_D1_ECC: 0 error
lucky01: DDR3_P2_D1_PRS: 0
lucky01: DDR3_P2_D1_TMP: 22 C (72 F)
lucky01: DDR3_P2_D2_ECC: 0 error
lucky01: DDR3_P2_D2_PRS: 0
lucky01: DDR3_P2_D2_TMP: 22 C (72 F)
lucky01: DDR3_P2_E1_ECC: 0 error
lucky01: DDR3_P2_E1_PRS: 0
lucky01: DDR3_P2_E1_TMP: 22 C (72 F)
lucky01: DDR3_P2_E2_ECC: 0 error
lucky01: DDR3_P2_E2_PRS: 0
lucky01: DDR3_P2_E2_TMP: 22 C (72 F)
lucky01: DDR3_P2_F1_ECC: 0 error
lucky01: DDR3_P2_F1_PRS: 0
lucky01: DDR3_P2_F1_TMP: 21 C (70 F)
lucky01: DDR3_P2_F2_ECC: 0 error
lucky01: DDR3_P2_F2_PRS: 0
lucky01: DDR3_P2_F2_TMP: 22 C (72 F)
lucky01: ECC_STROM: 0
lucky01: FM_TEMP_SENS_IO: 21 C (70 F)
lucky01: FM_TEMP_SEN_REAR: 22 C (72 F)
lucky01: HDD0_PRS: 0
lucky01: HDD1_PRS: 0
lucky01: HDD_BP_PRS: 0
lucky01: IOH_THERMALERT_N: 0
lucky01: IOH_THERMTRIP_N: 0
lucky01: IRQ_P1_RDIM_EVNT: 0
lucky01: IRQ_P1_VRHOT: 0
lucky01: IRQ_P2_RDIM_EVNT: 0
lucky01: IRQ_P2_VRHOT: 0
lucky01: LED_BLADE_STATUS: 0
lucky01: LED_FPID: 0
lucky01: LED_MEZZ_FAULT: 0
lucky01: LED_MEZZ_TP_FLT: 0
lucky01: LED_SAS0_FAULT: 0
lucky01: LED_SAS1_FAULT: 0
lucky01: LED_SYS_ACT: 0
lucky01: MAIN_POWER: 0
lucky01: MEZZ_PRS: 0
lucky01: P0V75_DDR3_P1: 0.7644 Volts
lucky01: P0V75_DDR3_P2: 0.7644 Volts
lucky01: P12V_BP: 11.948 Volts
lucky01: P12V_CUR_SENS: 10.78 Amps
lucky01: P1V05_ICH: 1.0486 Volts
lucky01: P1V1_IOH: 1.078 Volts
lucky01: P1V1_VCCP_P1: 1.0192 Volts
lucky01: P1V1_VCCP_P2: 0.931 Volts
lucky01: P1V1_VTT_P1: 1.1368 Volts
lucky01: P1V1_VTT_P2: 1.1564 Volts
lucky01: P1V2_SAS: 1.2152 Volts
lucky01: P1V5_DDR3_P1: 1.5288 Volts
lucky01: P1V5_DDR3_P1_IMN: 5.13 Amps
lucky01: P1V5_DDR3_P2: 1.5386 Volts
lucky01: P1V5_DDR3_P2_IMN: 14.25 Amps
lucky01: P1V5_ICH: 1.5092 Volts
lucky01: P1V8_IOH: 1.813 Volts
lucky01: P1V8_P1: 1.7836 Volts
lucky01: P1V8_P2: 1.7836 Volts
lucky01: P1_PRESENT: 0
lucky01: P1_TEMP_SENS: 39.5 C (103 F)
lucky01: P1_THERMTRIP_N: 0
lucky01: P2_PRESENT: 0
lucky01: P2_TEMP_SENS: 37.5 C (100 F)
lucky01: P2_THERMTRIP_N: 0
lucky01: P3V3_SCALED: 3.2548 Volts
lucky01: P3V_BAT_SCALED: 3.102 Volts
lucky01: P5V_SCALED: 4.9405 Volts
lucky01: POWER_ON_FAIL: 0
lucky01: POWER_USAGE: 126 Watts (430 BTUs/hr)
lucky01: SAS0_FAULT: N/A
lucky01: SAS1_FAULT: N/A
lucky01: SEL_FULLNESS: 0
lucky01: VR_P1_IMON: 1.75 Amps
lucky01: VR_P2_IMON: 3.5 Amps
  • John Walsh

    Awesome post!u00a0

  • John Walsh

    Awesome post! u00a0