Storage · Software-defined

Software-defined storage — the decisions it hands back to you

Software-defined storage moves redundancy, caching and failover out of a controller pair and into software across ordinary servers. What it does not move is the consequence: your nodes, drives and network become the specification.

• Scale by node — Not by controller
• Network first — Rebuild traffic, not average
• SMB, NFS, iSCSI — File and block services
• Burn-in tested — 24 to 72 hours
• Standard x86 — No proprietary shelf

What changes

The intelligence moves to software. The consequences move to your nodes.

In a traditional array, redundancy, caching and failover live in a pair of controllers you buy from one vendor and replace from the same one. Software-defined storage takes those jobs into software running across ordinary servers. What you buy becomes nodes, drives and network — and the software decides how they are used.

That is genuinely better in the ways people say: capacity and throughput grow together, a node is a smaller and cheaper step than a new array, and the hardware underneath is replaceable. It is also less forgiving, and the reason is not subtle. An array's controller pair is engineered by its maker to survive the failures the array can have. In SDS, that engineering is your specification.

So the useful conversation about SDS is not about the software. It is about how many failure domains you have, what the network does during a rebuild rather than on an average Tuesday, and whether the drives you chose behave under the write pattern the software will actually produce.

How it arrives

Hyperconverged, disaggregated, or object at scale

• Hyperconverged — Storage in the same nodes as the compute, replicated across them, with no separate array to buy. Fewest boxes, and the shape that punishes a thin network specification hardest.
• Disaggregated storage nodes — A separate pool of storage servers serving block or file to the compute tier. Compute and capacity scale on their own schedules, and a storage problem is not also a compute problem.
• Object at scale — S3 for archives that keep growing, erasure coded across nodes. This is where SDS is least contentious and most obviously right — see Spectra Object Storage for ours.

What the pool gives you

Shares, LUNs, and the four features that decide the bill

The services an estate actually consumes, and the four features that decide whether the pool is fast enough and small enough to pay for.

• SMB, NFS and iSCSI — File shares for users and applications, and thin-provisioned block targets for hypervisors and databases — one pool presenting all three rather than three systems to keep in step.
• Thin provisioning — Volumes sized for what the application believes it has, consuming only what it has written. Capacity is bought when it is used rather than when it is promised.
• Snapshots — Point-in-time copies taken in seconds and kept to a schedule, so a bad deployment or a deleted share is a rollback rather than a restore from tape.
• Cache drives for read and write — Flash in front of capacity drives, absorbing the write bursts and serving the hot reads, so a pool built on large spinning drives still answers quickly where it matters.
• Volume compression — Compression per volume rather than per pool, so the datasets that compress well do, and the ones that do not are not taxed for trying.
• Designed as required — Node count, drive mix, cache size and network are specified for the workload. There is no standard configuration to pick, because the right pool for an archive and the right pool for a database are not the same machine.

Where it goes wrong

Five decisions the software hands back to you

Every one of these is a hardware decision the software hands back to you. We work through them before quoting.

• Failure domains — What goes wrong when it is guessed: Redundancy spread across too few nodes tolerates drive loss but not node loss, and nobody notices until a node goes.
• Network for rebuild — What goes wrong when it is guessed: Sized for steady state, a cluster rebuilds slowly and stays degraded for days — the window in which a second failure is fatal.
• Drive choice and count — What goes wrong when it is guessed: Too few devices and the software cannot spread load; the wrong recording technology and sustained rewrites collapse.
• Memory and CPU on storage nodes — What goes wrong when it is guessed: SDS is software: checksums, erasure coding and replication cost cycles and RAM, and a node starved of both is slow in ways that look like disk.
• Growth step — What goes wrong when it is guessed: A cluster that must grow three nodes at a time is a budget problem, not a technical one — the step is worth choosing deliberately.

Our side of it

Hardware specified for the software that will run on it

• Validated platforms — Our rack and storage platforms are validated for Proxmox VE, VMware ESXi, XCP-ng, Hyper-V and Nutanix AHV, so the stack decision and the hardware decision stay separate.
• We run this ourselves — SpectraCloud is our own production cloud on Proxmox VE across more than 800 physical servers, so the advice on rebuild windows and node counts comes from operating them.
• What the burn-in is looking for — CPU, memory and storage stress with thermal checks and a SMART review. A marginal drive in an SDS cluster is a rebuild waiting to happen.
• Firmware baseline recorded — The revisions a cluster was validated on are written down, which is what keeps the node you add next year compatible with the nodes you have.
• Drives chosen for the write pattern — Sustained rewrite, write-once archive and mixed primary are three different duty cycles, and the largest drive on the market is right for exactly one of them.
• Sized together with backup — Replication is not backup. A cluster without a separate copy is one operator mistake from an outage, and we size the target alongside the pool.

Straight answers

Replication is not backup, and four other answers

• Is SDS cheaper than an array? — Answer: Per terabyte, usually. Per hour of engineering, not always. The saving is real and it moves cost from the purchase order to the specification, which is a good trade only if the specification gets the attention.
• How many nodes to start? — Answer: Enough that losing one leaves the redundancy scheme intact — which depends on the scheme, not on a rule of thumb. Three is the usual floor for hyperconverged; object storage starts at four in our own platform.
• Does replication replace backup? — Answer: No. Replication copies your mistakes faithfully and immediately. Retention and immutability are separate requirements and need a separate target.
• Can we mix drive sizes? — Answer: Technically usually yes, practically it wastes money: most implementations treat every drive as though it were the smallest one present.
• Who supports it? — Answer: We do. The warranty span and the response terms are the same for every machine we build, and they are set out in full on the services page.

Run the numbers

Two calculators that test an optimistic estimate

• RAID capacity planner — Usable capacity, redundancy overhead and rebuild exposure across RAID levels.
• VM sizing calculator — What a host really holds once reserves and ratios are honest.
• All sizing tools — Six calculators running the same arithmetic our engineers do.

Where it lands

The platforms behind an SDS pool

Nodes, drives and the storage that sits beside them.

• Spectra Object Storage — S3 to 4PB, erasure coded, from four storage nodes upward.
• Proxmox VE platforms — The cluster shapes we build, from the estate we run on it.
• SNS flash storage — All-flash shared storage for clusters that keep the array separate.