Virtualisation · Proxmox VE

NetBytes platforms for Proxmox VE — from the people running 800+ of them

SpectraCloud runs on Proxmox VE across more than 800 physical servers. These are the cluster shapes we build for it: single node, three nodes with shared storage, or three nodes hyperconverged with Ceph — validated, burn-in tested and specified from the workload.

• 800+ hosts — On Proxmox, in our own cloud
• 3-node minimum — For quorum and HA
• KVM and LXC — VMs and containers
• ZFS, Ceph, shared — Storage laid out to suit
• Burn-in tested — 24 to 72 hours

Why ask us

We run 800+ Proxmox hosts of our own

SpectraCloud is our own production cloud, and it runs on Proxmox VE across more than 800 physical servers. That is not a reference customer or a pilot — it is our own infrastructure, our own money, and our own team on call at three in the morning when a node stops.

It changes what we can tell you. Most of what is worth knowing about running Proxmox at scale is not in the documentation: how a cluster behaves when the network partitions rather than when a node dies cleanly, what a corosync ring wants from the switch underneath it, how long a Ceph rebalance really takes on the drives you were about to buy. We have the answers because we have had the outages.

So the platforms on this page are not a catalogue that happens to boot Proxmox. They are the shapes we run.

• Proxmox hosts we operate
• nodes for quorum
• burn-in before dispatch

The platform

No socket licence, no edition ladder, no exit fee

Proxmox VE is an open-source virtualisation platform built on Debian, running virtual machines through KVM and containers through LXC from one interface. Clustering, live migration, high availability, software-defined storage and backup are part of the platform rather than separate products with separate licences.

That last point is usually what starts the conversation. There is no per-socket or per-core licence to renew, and no edition ladder where the feature you need sits one tier above the one you bought. A support subscription is available and most production estates take one, but it is priced per node and it does not gate functionality.

The second reason is exit. The disk format is qcow2 or raw on ordinary filesystems, the configuration is text, and the hardware underneath is standard x86. A platform you can leave is a platform you can negotiate with.

Cluster shapes

The three layouts we build, and when each is right

Nearly every Proxmox deployment we ship is one of these. The difference between them is where the storage lives, and that decides almost everything else.

• Single node — One host, local ZFS, no cluster. Right for a branch office, a test estate or a first server room where the honest answer is that downtime for an hour is survivable. No quorum, no HA, and no pretending otherwise.
• Three nodes with shared storage — Three hosts for quorum, with the VM disks on a shared array over iSCSI or NFS. Compute and storage scale separately, the array is the thing you buy well, and a host failure is a restart elsewhere rather than a restore.
• Three nodes, hyperconverged with Ceph — Storage in the same chassis as the compute, replicated across nodes. Fewer boxes and no separate array, at the cost of needing the network and the drive layout to be right — this is the shape that punishes a thin specification.

Specifying it

What each shape needs you to decide

The questions below are the ones that change the bill of materials. We work through them before quoting, because a Proxmox cluster that is wrong is usually wrong in the network or the storage rather than in the CPU.

• Quorum — Single node: None — accept the restart. Three nodes, shared storage: Three votes; a witness if you ever run two. Three nodes, Ceph: Three votes, and keep it odd as you grow
• VM storage — Single node: Local ZFS or LVM-thin. Three nodes, shared storage: iSCSI or NFS from the array. Three nodes, Ceph: Ceph across the nodes, NVMe or a hybrid tier
• Network — Single node: One bonded pair is enough. Three nodes, shared storage: Separate storage and cluster traffic. Three nodes, Ceph: Separate, and size it for rebuild traffic, not steady state
• What a node failure costs — Single node: An outage until it is back. Three nodes, shared storage: A restart on a surviving node. Three nodes, Ceph: A restart, plus a rebalance you must have planned for
• Where it goes wrong — Single node: No backup, no second copy. Three nodes, shared storage: The array becomes the single point. Three nodes, Ceph: Undersized network or too few OSDs

Under the cluster

What a node has to get right

• Sized to the workload — Cores, memory, storage tier, network and accelerators chosen for the estate, not a fixed SKU. Our VM sizing calculator runs the same arithmetic we do before quoting.
• Validated before it ships — Every platform is validated for Proxmox VE alongside VMware ESXi, XCP-ng, Hyper-V and Nutanix AHV, so a hypervisor decision taken later does not become a hardware decision.
• What the burn-in is looking for — CPU, memory and storage stress with thermal checks and a SMART review, per node, before the cluster is validated as one.
• Firmware baseline recorded — The BIOS and controller revisions a cluster was validated on are written down, which is what makes the fourth node match the first three a year later.
• Backup planned with it — Proxmox Backup Server, incremental and deduplicated, sized alongside the cluster rather than after it — a cluster without a backup target is not finished.
• Built, tested and supported here — Integrated and manufactured in India, burn-in tested before dispatch, and covered by the warranty and response terms set out on the services page — the same for every machine we build.

Straight answers

Two nodes, Ceph, and moving from VMware

• Do we need a subscription? — Answer: Not to run it. The subscription buys the enterprise repository and support, is priced per node, and gates no features. Most production estates take it; test clusters usually do not.
• Is two nodes enough for HA? — Answer: No. Two nodes cannot hold quorum when they disagree, so a partition stops both. Three is the practical minimum, and a third vote can be a small witness rather than a full host.
• Ceph or a shared array? — Answer: Ceph if you want no separate array and can specify the network properly. A shared array if you would rather buy one good box and keep the failure domain obvious. We have run both at scale and neither answer is universal.
• Can we move existing VMware VMs? — Answer: Usually yes — Proxmox imports OVF and disk images, and qcow2 or raw conversion is routine. The work is in the network and the backup plan, not in the disks.
• Do you support it after it ships? — Answer: We do. The warranty span and the response terms are the same for every machine we build, and they are set out in full on the services page.

Do the arithmetic

Size it yourself before anyone quotes you

• VM sizing calculator — Cores, memory and the reserves that decide how many VMs a host really holds.
• How many VMs fit on one host — The guide the calculator is built from, including what to subtract before you start.
• All sizing tools — Six calculators running the same arithmetic our engineers do.

What to buy

Three nodes, and what goes in them

Three nodes, and the storage decision underneath them.

• Virtualization appliances — Pre-racked, pre-cabled three-node clusters, delivered configured.
• 2U rack servers — The usual cluster node — dual socket, NVMe and SAS tiers, redundant power.
• SNS flash storage — All-flash shared storage for the clusters that keep the array separate.