A little misadventure happened to me recently. After changing the IPMI password on my 1-node, I couldn’t log back into my node’s management interface. Forgot it instantly? Typo? Botched copy-paste? Whatever the reason, the access lockout was very real.

Losing your IPMI password (or BMC) is enough to make anyone break a sweat. But rest assured, experience has taught me a golden rule: as long as you still have access to the host operating system (whether it’s AHV, a standard Linux, or even ESXi), you can regain control without needing to perform a hardware reset.

However, the situation can be more critical than it seems. For the experts reading this, you know that in a Nutanix Single-Node environment, IPMI isn’t just an administrative convenience: it’s your one and only fail-safe. Without it, you can’t mount a Phoenix ISO via Remote Media during a major AOS crash or force a reboot. Losing BMC access on such a node quickly becomes a critical bottleneck for your production and remote console access.

In this guide, I’ll share the different methods to regain control of your node in a matter of minutes, from the gentlest approach to the last resort.

Method 1: In-band Reset via ipmitool

If you are running Nutanix AHV or a classic Linux distribution, you’re in luck: the ipmitool utility is usually available natively. The trick here relies on the KCS (Keyboard Controller Style) hardware channel. Simply put, this allows the host OS to communicate directly with the BMC chip via the motherboard bus (LPC/PCIe), entirely bypassing the management network.

Identify and Overwrite the Admin Account

Log in as root on your host. As a precaution, first ensure that the kernel modules managing IPMI are loaded:

modprobe ipmi_si
modprobe ipmi_devintf

Next, we’ll list the configured users (usually on channel 1) to spot the ID of our administrator account. In 99% of cases, the default account (ADMIN) corresponds to ID 2.

ipmitool user list 1

In my case, the ID is indeed 2:

Once the ID is confirmed, the command to force a new password is child’s play:

ipmitool user set password 2 'MyNewPwd123!'

When entering the previous command, the terminal might throw a nasty Invalid data field in request error.

Don’t panic; it’s simply the security policy parser of some firmwares rejecting your request. Not only does it require a minimum complexity (8 characters, uppercase, lowercase, number), but it can also be picky about the request format.

To bypass (or rather, satisfy) this mechanism, force the encoding length to 20 bytes by simply appending 20 to the end of your command:

ipmitool user set password 2 'MyNewPwd123!' 20

Your password has now been overwritten. You can test the connection to the web interface.

Method 2: The IPMICFG Utility (The Solution for VMware ESXi)

If you are running VMware ESXi, the first method might leave you stranded. Why? Because ipmitool is not packaged natively, and resolving dependencies to compile it on a hypervisor is a waste of time that I wouldn’t recommend.

The workaround is called IPMICFG. This is the official tool provided by Supermicro (downloadable from their portal with a free account). Its massive advantage is that it comes as static binaries, ready to use for Linux, Windows, and crucially, ESXi.

Once the archive is transferred to your host and extracted, using it is very straightforward:

./IPMICFG-Linux.x86_64 -user list
./IPMICFG-Linux.x86_64 -user setpwd 2 MyNewPwd123!

Note: Adapt the binary name if you are using the Windows or ESXi version.

You can download the utility here: https://www.supermicro.com/wdl/utility/IPMICFG/

Extreme Case: The Node No Longer Has a Bootable OS

What if the OS won’t boot at all? This is usually where the sweating starts, thinking that without network access to the IPMI or a functional OS, the only remaining option is a factory reset.

Remember: the KCS interface is exposed at the hardware level as soon as the BIOS POST completes. It is completely independent of your storage’s state.

All you have to do is boot the server from a USB stick containing a Linux Live CD (Ubuntu, Rocky Linux). Once on the live desktop, open a terminal, install the utility if it isn’t already there (e.g., apt install ipmitool), and run the exact same commands from Method 1.

Last Resort Method: The BMC Factory Reset

If for some obscure reason the previous methods fail, there is one ultimate resort left: resetting to factory settings.

With the Supermicro tool, the command is direct:

./IPMICFG-Linux.x86_64 -fd

And if you prefer the pure ipmitool method, you need to send a RAW command:

ipmitool raw 0x3c 0x40

⚠️ Warning, the consequences are heavy: Your entire BMC configuration is wiped clean. You will lose the static IP (the port reverts to DHCP), VLAN configurations, and user accounts. The BMC will then reboot (expect a good 2 to 3 minutes). Pay close attention to recent generations of servers: the password may not revert back to ADMIN/ADMIN. It defaults to the unique password printed on the OCP label (often stuck on the chassis or motherboard). Keep this in mind if your server is physically located on the other side of the world!

My Recommendations

Once access is recovered via method 1 or 2, here is what I systematically do to cover my back:

1. The Password Manager: It’s basic, but save your creds immediately instead of keeping them in your copy-paste clipboard.
2. The Backup Account: Create a second administrator account.
3. The Slot ID Trick: Do not put your backup account on Slot ID 3. Force its creation on a higher Slot (like 5 or 6). Why? Because some infrastructure automation scripts have the annoying tendency to forcefully overwrite or modify the early Slots (2 and 3) during provisioning. By placing it further down, you safeguard it.

What about you? Have you ever locked yourself out of one of your nodes? Share your worst nightmare in the comments below!