Proxmox + Kubernetes

Proxmox is a common home for Kubernetes: each Kubernetes node is a VM, and the Proxmox cluster supplies the machines, disks and networks. Kubernetes doesn't know that by itself. Two add-ons connect the two: a Cloud Controller Manager tells Kubernetes which VM and which physical host each node is, and a CSI plugin turns PersistentVolumeClaims into real Proxmox disks.

which distribution

VMs as Kubernetes nodes

All of these run fine in Proxmox VMs. Usually a template plus cloud-init (or Talos machine configs) creates them, through Terraform/OpenTofu or Cluster API. Spread control-plane VMs over different Proxmox nodes, otherwise losing one host loses etcd quorum.

	Talos Linux	kubeadm	k3s
OS	immutable, API-only, no SSH	any distro (Debian/Ubuntu)	any distro, single binary
Provisioning	ISO/nocloud image + machine config	cloud-init + kubeadm init/join	cloud-init + install script
Upgrades	talosctl upgrade (A/B image)	kubeadm upgrade per node	replace binary / system-upgrade-controller
Fits	production fleets, GitOps	learning the parts, full control	small clusters, homelabs, edge

sergelogvinov/proxmox-cloud-controller-manager

Cloud Controller Manager: who is this node?

Kubelets started with --cloud-provider=external register with a taint and wait for a CCM to initialize them. The Proxmox CCM sets providerID: proxmox://<region>/<vmid>, the topology labels (region = Proxmox cluster name from its config, zone = Proxmox node hosting the VM), an instance type and the node addresses. Its node-lifecycle controller deletes Node objects whose VM is gone.

Proxmox cluster “pve-dc1” → Kubernetes Nodes

Proxmox CCM running

region = Proxmox cluster · zone = Proxmox node · click a VM

pve1zone

pve2zone

pve3zone

The CCM asks the Proxmox API which VM a Node runs on (by providerID, or by matching name/UUID on first contact) and writes down what the scheduler needs: where the VM is (region/zone), its size and its IPs. It also removes the taint so pods can be scheduled.

apiVersion: v1
kind: Node
metadata:
  name: worker-2
  labels:
    topology.kubernetes.io/region: pve-dc1
    topology.kubernetes.io/zone: pve2
    node.kubernetes.io/instance-type: 4VCPU-8GB
spec:
  providerID: proxmox://pve-dc1/202
status:
  addresses:
  - type: InternalIP
    address: 10.20.0.152
  - type: Hostname
    address: worker-2

sergelogvinov/proxmox-csi-plugin

CSI: a claim becomes a disk

The Proxmox CSI plugin uses the Proxmox API. Its controller creates a disk on a Proxmox storage and hot-plugs it into the VM running the pod as a SCSI disk. A node plugin (a DaemonSet in each VM) formats and mounts it. With node-local storage such as local-lvm or local ZFS the disk exists on one host only, so the volume is zonal: the plugin reads the CCM's zone label to know where to create the disk and where the pod may run.

A PersistentVolumeClaim becomes a disk

pve1

worker-1

VM 201

local-lvm

pve2

worker-2

VM 202

local-lvm

pve3

worker-3

VM 203

local-lvm

1kubectl
PVC data-postgres-0 (10Gi, storageClass proxmox-local-lvm) created: Pending, because of WaitForFirstConsumer
2scheduler
Pod postgres-0 placed on worker-2 → its zone is pve2
3CSI controller
CreateVolume via the Proxmox API: a new 10G disk on storage local-lvm of node pve2
4CSI controller
ControllerPublish: hot-plug the disk into VM 202 as a new scsiN device (like qm set)
5CSI node plugin
inside worker-2: find the new SCSI disk, mkfs.ext4 on first use, mount into the pod
6kubelet
postgres-0 Running · PV has nodeAffinity zone=pve2

StorageClass for the Proxmox CSI plugin

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: proxmox-local-lvm
provisioner: csi.proxmox.sinextra.dev
parameters:
  storage: local-lvm          # Proxmox storage ID
  csi.storage.k8s.io/fstype: ext4
  cache: none
  ssd: "true"
volumeBindingMode: WaitForFirstConsumer   # pick the pod's node first, then create the disk there
allowVolumeExpansion: true
reclaimPolicy: Delete

Why WaitForFirstConsumer matters

With immediate binding the disk would be created before the scheduler has chosen a node, maybe on pve1 while the pod later lands on pve3. Waiting for the first consumer means: choose the pod's node, then create the disk on that host's storage.

Zonal volumes and maintenance

A pod using a local-storage volume can only run on VMs in that Proxmox node's zone. Run several replicas spread across zones (topologySpreadConstraints) and let the application replicate the data (Postgres streaming, Kafka, etcd), or use shared storage.

pveum

API tokens with minimal privileges

Give each component its own user and token, scoped to what it does. The CCM only reads VM and node information. The CSI plugin also allocates storage and edits VM disk configuration. Roles below follow the projects' install docs; check them for your plugin version, since features like replication or volume migration need more privileges. In production, scope the ACL to the relevant pools and storages instead of /.

on any Proxmox node

# --- Cloud Controller Manager: read-only ---
pveum role add CCM -privs "VM.Audit VM.GuestAgent.Audit Sys.Audit"
pveum user add kubernetes@pve
pveum aclmod / -user kubernetes@pve -role CCM
pveum user token add kubernetes@pve ccm -privsep 0

# --- CSI plugin: may create disks and change VM disk config ---
pveum role add CSI -privs "VM.Audit VM.Config.Disk Datastore.Allocate Datastore.AllocateSpace Datastore.Audit"
pveum user add kubernetes-csi@pve
pveum aclmod / -user kubernetes-csi@pve -role CSI
pveum user token add kubernetes-csi@pve csi -privsep 0

cloud config (Kubernetes Secret)

# CSI config.yaml (the CCM uses the same format with its own token)
clusters:
  - url: https://pve1.example.com:8006/api2/json
    insecure: false
    token_id: "kubernetes-csi@pve!csi"
    token_secret: "xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
    region: pve-dc1          # becomes topology.kubernetes.io/region

privsep

With -privsep 0 the token has its user's privileges. With -privsep 1 it has no privileges until you also grant an ACL to the token itself (pveum aclmod / -tokens 'kubernetes-csi@pve!csi' -role CSI).

shared instead of zonal

Alternative: Ceph RBD CSI on Proxmox's Ceph

If the Proxmox cluster runs Ceph, Kubernetes can use it directly with the upstream ceph-csi RBD driver. The VMs map RBD images over the network, so volumes are not tied to a host. A pod can restart on any node and Ceph replicates the data. In exchange, the Kubernetes VMs need a route to the Ceph public network and a Ceph key, so Kubernetes now depends directly on the storage cluster. Switch the CSI demo above to ceph-csi to compare.

prepare Proxmox Ceph for ceph-csi

# on a Proxmox node: a pool and a restricted Ceph user for Kubernetes
pveceph pool create kubernetes --application rbd
ceph auth get-or-create client.kubernetes \
  mon 'profile rbd' osd 'profile rbd pool=kubernetes' mgr 'profile rbd pool=kubernetes'
ceph fsid                     # → clusterID for ceph-csi
ceph mon dump                 # → monitor addresses

	Proxmox CSI	ceph-csi RBD
Talks to	Proxmox API (token)	Ceph MONs/OSDs (cephx key)
Disk appears as	SCSI disk hot-plugged by QEMU	/dev/rbdX mapped inside the VM
Topology	zonal for local storage	any node
Network	none extra	VMs must reach Ceph public net
Needs	CCM zone labels	nothing from CCM

why it mattersThe CCM tells Kubernetes where each node is: providerID proxmox://<cluster>/<vmid>, region = Proxmox cluster, zone = Proxmox host. The scheduler, the CSI plugin and your topology spread rules all rely on these labels.
why it mattersThe Proxmox CSI plugin turns a PVC into a real VM disk through the API and hot-plugs it into the right VM. With local storage that volume is zonal, so plan replicas per Proxmox host.
why it mattersUse separate API tokens with minimal roles for CCM (audit only) and CSI (disk + datastore). Choose ceph-csi when pods must move freely between hosts and the VMs can reach Ceph.