Planning a Proxmox VE Cluster: Quorum, Storage and Network Layout

Proxmox VE puts KVM virtual machines and LXC containers behind one management layer, backed by a clustered configuration filesystem. Most of the difficulty in running it is not the installer. It is deciding, before the first guest exists, how storage, quorum and networking will be laid out, because those choices are painful to reverse once workloads are running.

Virtual machines or containers

A KVM guest emulates a full machine and boots its own kernel. An LXC container shares the host kernel, so it starts faster and uses far less memory, but it cannot run a different kernel. Anything needing custom kernel modules, nested virtualization, or an unusual filesystem driver belongs in a VM. Prefer unprivileged containers, which map container root to an unprivileged range on the host. The usual mistake is switching a container to privileged because a bind mount was easier that way, which quietly removes the isolation boundary you were relying on.

Storage decisions come first

Proxmox supports local directory storage, LVM and LVM-thin, ZFS, and shared backends such as NFS, iSCSI and Ceph RBD. The important property is not raw speed. It is what each backend allows you to do later. Not every storage type supports snapshots, and live migration between nodes with no downtime generally requires storage both nodes can reach, or replicated ZFS datasets. If you build every node on isolated local storage and later want to move a guest during maintenance, you will be doing a cold migration.

Ceph is attractive because it turns the same nodes into a distributed storage pool with no separate array. It also has real requirements: enough nodes to satisfy your replication rules, dedicated fast networking, and headroom so that a failed node can be recovered onto the remaining ones. Running Ceph on two nodes, or on the same link as everything else, is where most homelab Ceph disappointment comes from.

Quorum is the part people underestimate

Cluster membership is handled by corosync, and a cluster needs a majority of votes to stay quorate. A two-node cluster has no majority once either node drops, so the survivor stops being authoritative. The fix is a third vote, either another node or a lightweight quorum device. Without quorum the cluster filesystem holding your guest configuration goes read only, and you cannot start or edit guests even though the hardware is fine.

Corosync is latency sensitive. Give it a link that is not competing with backup jobs, storage replication or bulk VM traffic. Token timeouts caused by a saturated shared link look exactly like a hardware failure and can trigger fencing on a healthy node.

Network layout

Guests attach to Linux bridges. Make the bridge VLAN aware early if you expect to segment traffic, because retrofitting it means touching every guest. Separate management, cluster communication, storage and guest traffic where you have the interfaces to do so. For PCIe passthrough, IOMMU must be enabled in firmware and on the kernel command line, and the device needs to be isolated in its own IOMMU group. Group layout is a property of the board and chipset, so confirm it before buying hardware for a passthrough build.

Backups and testing

Built in backup jobs are straightforward to schedule. The failure mode is storing them on the same pool as the guests they protect. Keep at least one copy on separate hardware, and restore something occasionally. An untested backup is an assumption, not a recovery plan.