xCAT out of the box works on UCS. Or UCS out of the box works with xCAT? Whichever way you look at it, it works. All of the cool things you can do with xCAT like provision nodes, KVM, vSphere, stateless computing, etc, can all be done with UCS. In fact, you can even run most of the r* commands on UCS.
Cisco UCS allows this through IPMI. And configuring IPMI on UCS is easier than any other system I’ve ever used. While I still plan on furthering my xCAT UCS plugin to get more capabilities into xCAT, most xCAT functions can be used with UCS managing the servers with IPMI. For most people, this is good enough.
Using IPMI this is what seems to work with xCAT 2.6.6 and UCSM 2.0(1): (See the end of this for sample output)
- rpower on|off|stat|boot
- rbeacon on|off
- reventlog [clear]
- rvitals (this is quite thorough)
rinv seems to hang on me. This I think is due to the nature of service profiles, where UUIDs and MAC addresses are transient. I’ll investigate this further.
So how do you do it?
Configuring an IPMI machine with xCAT has been well documented. What I haven’t seen documented so much is configuring IPMI inside UCS. This is surprisingly easy. Here’s how its done:
1. Create a Service Profile Template that you will apply to your blades. This is documented very well in various places so I won’t go into it here. Creating a Service Profile Template is UCS 101. After you’ve created your service profile, assuming its an updating template you can proceed to the next step. (Don’t worry, any changes made for doing IPMI don’t require a reboot)
2. From the Servers tab, filter by Service Profile Templates, and navigate to your service profile template.
3. Click on the policies table and look at the IPMI Access Profile Policy
4. Create a new policy. In this policy you’ll give the name of the user and give it a password. Make sure they have admin privileges. For simplicity, I just made my user and password the same as my UCSM user and password.
5. Apply the setting and click save.
From here on out you can just run IPMI commands. The only issue now is to know which IP address corresponds to the IPMI interface of which blade.
Ok my friend, you now have it. xCAT running rpower commands.
And now, here is a sample output running rvitals on a UCS B200 M1:
# rvitals lucky01 lucky01: BIOSPOST_TIMEOUT: N/A lucky01: BIOS_POST_CMPLT: 0 lucky01: CATERR_N: 0 lucky01: CPUS_PRCHT_N: 0 lucky01: DDR3_P1_A1_ECC: 0 error lucky01: DDR3_P1_A1_PRS: 0 lucky01: DDR3_P1_A1_TMP: 26 C (79 F) lucky01: DDR3_P1_A2_ECC: 0 error lucky01: DDR3_P1_A2_PRS: 0 lucky01: DDR3_P1_A2_TMP: 25 C (77 F) lucky01: DDR3_P1_B1_ECC: 0 error lucky01: DDR3_P1_B1_PRS: 0 lucky01: DDR3_P1_B1_TMP: 26 C (79 F) lucky01: DDR3_P1_B2_ECC: 0 error lucky01: DDR3_P1_B2_PRS: 0 lucky01: DDR3_P1_B2_TMP: 27 C (81 F) lucky01: DDR3_P1_C1_ECC: 0 error lucky01: DDR3_P1_C1_PRS: 0 lucky01: DDR3_P1_C1_TMP: 24 C (75 F) lucky01: DDR3_P1_C2_ECC: 0 error lucky01: DDR3_P1_C2_PRS: 0 lucky01: DDR3_P1_C2_TMP: 25 C (77 F) lucky01: DDR3_P2_D1_ECC: 0 error lucky01: DDR3_P2_D1_PRS: 0 lucky01: DDR3_P2_D1_TMP: 22 C (72 F) lucky01: DDR3_P2_D2_ECC: 0 error lucky01: DDR3_P2_D2_PRS: 0 lucky01: DDR3_P2_D2_TMP: 22 C (72 F) lucky01: DDR3_P2_E1_ECC: 0 error lucky01: DDR3_P2_E1_PRS: 0 lucky01: DDR3_P2_E1_TMP: 22 C (72 F) lucky01: DDR3_P2_E2_ECC: 0 error lucky01: DDR3_P2_E2_PRS: 0 lucky01: DDR3_P2_E2_TMP: 22 C (72 F) lucky01: DDR3_P2_F1_ECC: 0 error lucky01: DDR3_P2_F1_PRS: 0 lucky01: DDR3_P2_F1_TMP: 21 C (70 F) lucky01: DDR3_P2_F2_ECC: 0 error lucky01: DDR3_P2_F2_PRS: 0 lucky01: DDR3_P2_F2_TMP: 22 C (72 F) lucky01: ECC_STROM: 0 lucky01: FM_TEMP_SENS_IO: 21 C (70 F) lucky01: FM_TEMP_SEN_REAR: 22 C (72 F) lucky01: HDD0_PRS: 0 lucky01: HDD1_PRS: 0 lucky01: HDD_BP_PRS: 0 lucky01: IOH_THERMALERT_N: 0 lucky01: IOH_THERMTRIP_N: 0 lucky01: IRQ_P1_RDIM_EVNT: 0 lucky01: IRQ_P1_VRHOT: 0 lucky01: IRQ_P2_RDIM_EVNT: 0 lucky01: IRQ_P2_VRHOT: 0 lucky01: LED_BLADE_STATUS: 0 lucky01: LED_FPID: 0 lucky01: LED_MEZZ_FAULT: 0 lucky01: LED_MEZZ_TP_FLT: 0 lucky01: LED_SAS0_FAULT: 0 lucky01: LED_SAS1_FAULT: 0 lucky01: LED_SYS_ACT: 0 lucky01: MAIN_POWER: 0 lucky01: MEZZ_PRS: 0 lucky01: P0V75_DDR3_P1: 0.7644 Volts lucky01: P0V75_DDR3_P2: 0.7644 Volts lucky01: P12V_BP: 11.948 Volts lucky01: P12V_CUR_SENS: 10.78 Amps lucky01: P1V05_ICH: 1.0486 Volts lucky01: P1V1_IOH: 1.078 Volts lucky01: P1V1_VCCP_P1: 1.0192 Volts lucky01: P1V1_VCCP_P2: 0.931 Volts lucky01: P1V1_VTT_P1: 1.1368 Volts lucky01: P1V1_VTT_P2: 1.1564 Volts lucky01: P1V2_SAS: 1.2152 Volts lucky01: P1V5_DDR3_P1: 1.5288 Volts lucky01: P1V5_DDR3_P1_IMN: 5.13 Amps lucky01: P1V5_DDR3_P2: 1.5386 Volts lucky01: P1V5_DDR3_P2_IMN: 14.25 Amps lucky01: P1V5_ICH: 1.5092 Volts lucky01: P1V8_IOH: 1.813 Volts lucky01: P1V8_P1: 1.7836 Volts lucky01: P1V8_P2: 1.7836 Volts lucky01: P1_PRESENT: 0 lucky01: P1_TEMP_SENS: 39.5 C (103 F) lucky01: P1_THERMTRIP_N: 0 lucky01: P2_PRESENT: 0 lucky01: P2_TEMP_SENS: 37.5 C (100 F) lucky01: P2_THERMTRIP_N: 0 lucky01: P3V3_SCALED: 3.2548 Volts lucky01: P3V_BAT_SCALED: 3.102 Volts lucky01: P5V_SCALED: 4.9405 Volts lucky01: POWER_ON_FAIL: 0 lucky01: POWER_USAGE: 126 Watts (430 BTUs/hr) lucky01: SAS0_FAULT: N/A lucky01: SAS1_FAULT: N/A lucky01: SEL_FULLNESS: 0 lucky01: VR_P1_IMON: 1.75 Amps lucky01: VR_P2_IMON: 3.5 Amps