Compute in a cluster is disposable, which is the whole point. Storage is not, which is why storage decisions outlive the cluster that made them — and why a wrong one is expensive to undo six months later.
This is the practical map: what the primitives are, which one fits which workload, and the three settings that cause most of the damage.
A PersistentVolumeClaim is what your workload asks for: this much space, this access mode, this class. A StorageClass decides who fulfils it and how. A PersistentVolume is the resulting piece of storage, provisioned by a CSI driver talking to the platform.
In practice you write PVCs and pick a StorageClass. Everything else happens for you.
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: pgdata
spec:
accessModes: [ReadWriteOnce]
storageClassName: nvme
resources:
requests:
storage: 100GiBlock storage is a disk. One writer, low latency, filesystem semantics your database expects. On our managed Kubernetes this is the NVMe class, attached over the network and following the pod as it reschedules.
File storage is a shared filesystem that several pods can mount at once. Convenient, and consistently slower and more fragile than teams expect under concurrent write load.
Object storage is not a volume at all — it is an API your application calls. It is also the correct answer far more often than the other two.
emptyDir. It dies with the pod, which is what you want. Add medium: Memory if it needs to be fast and small.This is the detail that bites. ReadWriteOnce means the volume is mounted by one node — several pods on that same node can share it, and a pod scheduled elsewhere cannot start. ReadWriteMany means many nodes at once, and requires file storage that supports it. ReadOnlyMany is many readers, no writers.
The symptom of getting this wrong is a rollout that hangs forever: the new pod waits for a volume the old pod still holds on another node, and neither moves. With RWO volumes use strategy: Recreate rather than RollingUpdate, and accept the few seconds of downtime — it is a database, it was never going to roll cleanly.
A Deployment with a PVC gives every replica the same volume, which is wrong for anything that writes. A StatefulSet gives each replica its own, with a stable name that survives rescheduling.
apiVersion: apps/v1
kind: StatefulSet
spec:
serviceName: pg
replicas: 3
volumeClaimTemplates:
- metadata:
name: data
spec:
accessModes: [ReadWriteOnce]
storageClassName: nvme
resources:
requests:
storage: 100GiYou get data-pg-0, data-pg-1, data-pg-2. Note that deleting the StatefulSet does not delete those claims — which is usually a relief, occasionally a surprise on the invoice.
A StorageClass has a reclaimPolicy. With Delete, removing the PVC destroys the underlying volume and everything on it. With Retain, the volume survives for you to reattach or clean up manually.
Dynamic provisioning defaults to Delete, and it is correct for scratch and staging. For anything whose loss would ruin a week, use a class with Retain:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: nvme-retain
provisioner: csi.antyxsoft.cloud
parameters:
type: nvme
reclaimPolicy: Retain
allowVolumeExpansion: true
volumeBindingMode: WaitForFirstConsumerThe cost of Retain is orphaned volumes you have to tidy up. The cost of Delete is a kubectl delete -f in the wrong namespace taking the database with it.
With allowVolumeExpansion: true you can grow a volume by editing the PVC, and most drivers do it online. You cannot shrink it. Size for the next six months rather than the next three years, and monitor free space — a full volume takes the database down in a way that looks like a crash rather than a capacity problem.
kubectl patch pvc pgdata -p '{"spec":{"resources":{"requests":{"storage":"200Gi"}}}}'CSI snapshots are fast, cheap and excellent for the case where you are about to do something regrettable to a database. They also live in the same platform as the volume and share its failure modes and its credentials.
apiVersion: snapshot.storage.k8s.io/v1
kind: VolumeSnapshot
metadata:
name: pgdata-pre-upgrade
spec:
volumeSnapshotClassName: csi-nvme
source:
persistentVolumeClaimName: pgdataTake snapshots before upgrades and migrations. Keep the real backup — logical dumps or streaming replication, shipped to object storage with immutability and separate credentials — as the thing you would actually rely on after a bad day.
Network-attached storage has a latency floor. It is fine for the overwhelming majority of databases and completely wrong for etcd, which is why the control plane's storage is our problem rather than yours on a managed cluster.
IOPS are typically per volume, not per pod. Ten pods sharing one volume share its limit; ten pods with their own volumes do not. If a workload is IO-bound, more volumes often beats a bigger volume.
Databases get block storage, one volume per replica, through a StatefulSet, with Retain and Recreate. Uploads and media go to object storage, not a shared filesystem. Scratch goes to emptyDir, logs go to stdout. Size generously because shrinking is not a thing, and keep a real backup somewhere the cluster's credentials cannot reach.