A few years ago, following a power outage at one of our two sites, I had to deal with some application issues after the Nutanix cluster restarted. On paper, everything should have gone smoothly: the UPS had taken over, causing no impact on production, but that was without counting on a mishandling by the technician who turned off the UPS. The cluster shut down abruptly.

Once the power returned, the cluster restarted automatically, and all the VMs restarted shortly after. The problem was that some database servers took their time… Too much time for certain business applications…

It’s exactly to save you from these late-night cold sweats that we are going to explore VM Startup Policies on Nutanix AHV today.

While the concept of sequencing the startup of your servers might seem simple, the underlying mechanics are worth a closer look.

What are “VM Startup Policies”?

VM Startup Policies are a native feature of the AHV hypervisor that acts as a scheduler when powering on your virtual machines. This feature was added with version 7.5, which I talked about a while ago on the blog.

Instead of suffering the startup order and leaving control to the cluster, you define precise groups of VMs and strict dependency rules. This way, you guarantee that your infrastructure services (like an Active Directory, for example) and your databases are fully operational before allowing the application servers to start.

To fully understand the importance of these rules, a quick reminder of Acropolis HA’s default behavior is necessary. In the absence of a Startup Policy, during a node crash or a cluster restart, the restart of VMs is done in best-effort mode. The Prism Element cluster focuses on rapid resource allocation (CPU/RAM) and triggers the power-on of VMs completely asynchronously and concurrently, and natively has no awareness of your application dependencies.

When do these rules really activate?

It is important to clarify that these strategies do not apply if you decide to restart a VM manually from the Prism Central interface in the middle of the day.

AHV orchestration wakes up and applies your Startup Policies in two critical scenarios:

  1. A High Availability (HA) event: One or more physical nodes in your cluster fail. The AHV hypervisor will migrate and restart the affected VMs on the surviving nodes, strictly respecting the order you defined.
  2. A Full Cluster Restart: Exactly the case of my anecdote! Following a total power outage, for example.

Decrypting the Mechanics: The 3 Fundamental Pillars

Nutanix’s approach has the merit of being extremely visual and very easy to grasp from Prism Central. It relies on three fundamental concepts.

1. Categories

Nutanix’s mechanics rely exclusively on Prism Central Categories. You don’t link “SQL-Server-01” to “Web-Server-01”, but you link the AppMetier:Database category, for example, to the VMs hosting databases and your application VMs to the AppMetier:Applicatif category.

This is where the true power of this architecture lies. If tomorrow you deploy three new application servers in scale-out to handle an unexpected load, you just need to assign them the right category. They will automatically be integrated into the correct order of your Startup Policy without you having to open or modify a single rule.

2. Dependency Levels

This is how you draw your logical chain: category B can only start if category A has finished. There is still a small subtlety to keep in mind: Nutanix allows a maximum of 6 consecutive dependency levels per policy. In 99% of cases, this is more than enough for a classic application (e.g., Infra > DB > Web), but it requires grouping your services well without micromanagement.

3. Start Conditions

That’s the million-dollar question: how does the AHV hypervisor know that the database is really ready to authorize the next step? The tool offers you 2 options:

  • VM Power On: As soon as the VM is powered on at the hardware level, the next step is launched. The Windows or Linux OS is not even loaded yet. To be avoided for strong dependencies.
  • Guest Boot up: The best of the best! Nutanix waits patiently for the OS to be fully loaded and the network layer to respond.

For the “Guest Boot up” condition to work, the Nutanix Guest Tools (NGT) must absolutely be installed and up to date on the VM. My advice (and the vendor’s recommendation): do not skip the NGT deployment!

There is also an option that complements either one:

  • Delay (in seconds): We “blindly” wait a given time (e.g., 60 seconds) before moving on.

Tutorial: Creating a Startup Policy

Let’s model the most common example: we want to start our domain controllers, wait for them to be ready, start our databases, and finally our application servers.

For this occasion, I created 3 categories: AppType:ActiveDirectory, AppMetier:Database, and finally AppMetier:Applicatif

  1. Log in to Prism Central.
  2. Go to the Infrastructure > Compute > VMs > Policies > VM Startup Policies menu.
  3. Click on Create VM Startup Policy.
  4. Give it an explicit name (e.g., Policy-Tiering-Metier).
  5. In the visual interface, add your first category (e.g., AppType:ActiveDirectory).
  6. Add a second block for AppMetier:Database and a third for AppMetier:Applicatif.
  7. Click on + Configure Start Conditions to define the startup conditions. Choose Guest Boot up and optionally add a 30-second safety delay to allow the internal OS services time to initialize properly.

Click on “Create”, and you’re done!

All that’s left is to add your VMs to the right categories.

Pitfalls to Avoid

The tool is great, but here is some field feedback to help you avoid falling into certain traps:

1. Manage the 6-Level Limit Intelligently

Don’t fall into the micromanagement trap! If you try to make a policy for every micro-application, you will quickly saturate the interface, reach the famous 6-level limit, and end up with a convoluted system that’s impossible to maintain over time. Group your resources intelligently and keep it simple!

2. Beware of the Interaction with DR Mechanisms

Startup Policies manage the local resilience of a cluster (HA or reboot). But beware, if you use Nutanix Disaster Recovery to failover to a remote site, this system has its own recovery plan mechanism. During an inter-site failover, it’s the Prism Central Recovery Plan that takes control of the startup order, and no longer your local Startup Policy. Therefore, ensure logical consistency between the two tools.

3. This Only Applies to Guest VMs

There is no need to try sequencing “Nutanix” VMs (Prism Central, CVM, or others…), it will not work. The mechanism is only applicable to your virtual machines, not system ones.

Conclusion

VM Startup Policies on Nutanix AHV are the kind of feature you configure once, forget about, and which can save precious time and provide peace of mind the day the unthinkable happens. Thanks to an intuitive visual interface based on categories, Nutanix has made application resilience accessible to everyone.

If you haven’t already done so, I strongly encourage you to check the status of your NGT (deployment status and versions) and categorize your most critical VMs today. Your future “you,” woken up at 3 AM during the next production incident, will thank you!