Proxmox high availability: keeping virtual machines running when a node fails

A server fails at some point. The power supply dies, a disk gives out, the kernel panics at three in the morning. If every virtual machine you run lives on that one box, everything the business depends on goes down with it, and it stays down until someone drives in and fixes the hardware. Proxmox high availability is the feature that turns that outage into a brief hiccup — the failed node’s virtual machines start up automatically on a surviving one. It is genuinely useful, and it is also widely misunderstood. Here is what it does, how it works, and whether your business actually needs it.

What HA does, in one sentence

Proxmox HA watches your virtual machines, and when the node running one dies, it restarts that VM on another node in the cluster, automatically, without a human. That is the whole promise. Notice the word restart — the VM reboots on the new node, it does not teleport mid-flight with its memory intact. This distinction matters, and we will come back to it.

To deliver even that, several pieces have to be in place. HA is not a switch you flip on a single server; it is a property of a properly built cluster.

The building blocks

A cluster. HA requires multiple Proxmox nodes joined into a cluster, so that when one fails there is somewhere for its workloads to go. The nodes talk to each other constantly over a cluster network, using Corosync, to agree on who is alive and who is not. That agreement is the foundation everything else stands on.

Quorum. In a cluster, the dangerous failure is not a node dying — it is the network splitting, so that two halves of the cluster each think the other is dead and both try to run the same VM. That is called split-brain, and it corrupts data. Proxmox prevents it with quorum: the cluster only acts if a majority of nodes agree they can see each other. A minority partition knows it is the minority and refuses to run HA workloads. This is why an odd number of nodes matters, and why three is the practical minimum. With two nodes, a single failure leaves exactly half the cluster, which is not a majority — so a two-node HA setup needs a lightweight tiebreaker, a QDevice, running somewhere else to cast the deciding vote.

Storage the VMs can reach from any node. A VM can only restart on another node if that node can read the VM’s disk. There are two common ways to arrange this. Shared storage — a SAN, NFS, or a Ceph cluster — puts the disks in a place every node can reach, so any node can pick up any VM. Storage replication — Proxmox’s built-in ZFS replication — instead copies each VM’s disk to another node on a schedule, so a recent copy already exists where the VM would restart. Shared storage means no data loss on failover; replication is simpler and cheaper but means you can lose whatever changed since the last replication interval. Which one you choose shapes the cost and the guarantees of the whole design.

Fencing. Before the cluster restarts a VM elsewhere, it must be certain the original node is truly not still running it — because two copies of one VM writing to storage is the corruption scenario again. Proxmox handles this with a hardware watchdog. A node that loses quorum cannot assume it is healthy; its watchdog forces it to reset itself within a set time. Only after that guaranteed self-fence does the cluster restart the VM on a healthy node. Fencing is the unglamorous safety mechanism that makes automatic failover safe rather than reckless.

The HA manager. Tying it together is Proxmox’s HA stack, which watches the VMs you have marked as HA-managed, notices when a node stops responding, waits for fencing to complete, and starts those VMs on a surviving node according to the rules you set. You tell it which VMs matter; it does the rest.

What HA protects against

HA is designed for one failure mode: a node going down. Hardware death, a crash, a node you have to take offline for maintenance. In those cases HA moves the affected VMs to healthy hardware and gets them running again in minutes, unattended. For a business where an hour of downtime is expensive and failures at inconvenient times are inevitable, that is real value.

Proxmox also supports live migration for the planned case — moving a running VM to another node with no downtime at all, so you can patch or reboot a host during business hours without anyone noticing. Live migration is the graceful sibling of HA’s emergency restart.

What HA does not protect against

This is where expectations get corrected, and it is the more important half of the article.

• It is not backup. HA keeps VMs running across a hardware failure. It does nothing about a deleted file, ransomware, a corrupted database, or a botched update — those get faithfully kept running and, if replicated, faithfully copied to the other node. You still need real, tested backups. HA and backups solve different problems, and neither substitutes for the other. If you have not proven your restores lately, that is the more urgent gap.
• It does not prevent downtime, only shortens it. The VM reboots on the new node. Anything in memory is lost, in-flight transactions may need recovery, and the application comes back as if it had been power-cycled. Minutes of interruption, not zero. If you need truly uninterrupted service you are into application-level clustering, which is a larger undertaking.
• It does not fix application failures. HA watches the VM, not what runs inside it. A database that has hung while the VM stays up is invisible to it. Service-level health checks are a separate layer.
• Shared storage can become the single point of failure. If every node depends on one SAN or NAS and that device fails, HA has nowhere healthy to restart anything. This is why serious shared-storage HA uses something distributed and self-healing like Ceph, which removes the single point but adds real cost and operational complexity.

Is it worth it for a small business

Honestly, sometimes not. HA earns its keep when downtime is genuinely costly and frequent-enough hardware failure is a real risk you cannot absorb. A three-node cluster with replicated or distributed storage is more hardware, more network, and more to maintain than a single well-backed-up server, and for a business that can tolerate a few hours of downtime for the price of a spare-parts plan and good backups, that simpler setup is often the right and cheaper answer.

HA becomes worth it when the maths changes: when a line-of-business system going down stops the whole company from working, when after-hours failures cannot wait for someone to drive in, or when a compliance or customer commitment makes unplanned downtime unacceptable. At that point the cluster pays for itself the first time a node dies at 3 a.m. and nobody has to wake up.

The right call depends on your actual tolerance for downtime and your budget, which is a conversation worth having before you buy hardware. It is exactly the kind of design work we do on the infrastructure side, often as part of a move from VMware to Proxmox, and we compared the two platforms for Atlantic Canada businesses in Proxmox vs VMware.

Send us two paragraphs about what your business runs and how long it could survive with a server down, and we will reply in writing within one business day.