Live migration

Live migration moves a running VM to another node with a pause short enough that TCP connections survive. The trick is pre-copy: copy all RAM while the guest keeps running, then re-copy only what it changed meanwhile, again and again, until the leftover is small enough to send during a pause of about 100 ms. It works when the link drains memory faster than the guest dirties it, and only then.

Pre-copy live migration

VM RAM

migration link

guest dirty rate300 MiB/sworking set25%auto-converge

guest RAM on source · 1 cell = 32 MiBrunning on source

to send this round copied, clean dirtied after copy working set

iteration

–

elapsed

0.0 s

sent

0.0 GiB

est. downtime

–

vCPU throttle

dirty vs link

300 / 1120

MiB sent per iteration (should shrink)

dashed: what fits in 100 ms of downtime · amber bars: vCPUs throttled

Press ▶. The first iteration copies all of RAM while the guest keeps running and keeps writing.

the math

Why it converges, and when it doesn't

Each iteration lasts as long as it takes to send what the previous one left dirty. If the guest dirties d MiB/s and the link carries b MiB/s, each iteration's leftover is roughly d/b of the previous one: with d < b it shrinks geometrically, with d ≥ b it never does. Real guests help a bit: they rewrite the same hot pages (the working set), so dirtying saturates. When it still doesn't converge, QEMU's auto-converge (enabled by Proxmox) throttles the vCPUs, starting at 20% and stepping up, to slow the writer down, and Proxmox raises the allowed downtime step by step when the remaining amount stops shrinking. Try 1 GbE with a 300 MiB/s dirty rate, with and without auto-converge.

What actually moves

pve2 starts an empty, paused QEMU with the same config (incoming migration).

guest downtime: ≈ 10–300 ms

Shared storage is what makes it cheap

On Ceph, NFS or iSCSI, both nodes already see the disk; only RAM and device state travel. With local disks (--with-local-disks) QEMU's drive-mirror has to copy every allocated block first, so a 500 GiB disk turns a 10-second job into an hour. If the disk is ZFS-replicated to the target, only the delta since the last replication has to be mirrored.

Containers restart

LXC has no reliable live migration (checkpoint/restore of arbitrary processes is not production-ready), so pct migrate --restart shuts the CT down, moves it, and boots it again. Without --restart, only stopped CTs can be migrated.

Give migration its own network

By default migration traffic uses the cluster network. Set migration: network= in datacenter.cfg to a fast dedicated CIDR so a 128 GiB migration can't starve corosync. type=secure (default) tunnels through SSH; insecure sends RAM unencrypted for more speed: only on a trusted, isolated network.

cpu type

The guest must not notice a different CPU

CPU type decides where a VM may go

cpu: x86-64-v2-AES — Default for new VMs. A baseline (SSE4.2, POPCNT, AES-NI…) that every x86 server from the last ~12 years provides. A running guest can't lose CPU features mid-flight, so every migration target must offer at least what the guest was started with.

pve1VM runs here

AMD EPYC 9354 (Zen 4)

x86-64-v4

pve2✓ can migrate

AMD EPYC 9354 (Zen 4)

supports v2

pve3✓ can migrate

Intel Xeon Gold 6330 (Ice Lake)

supports v2

pve4✓ can migrate

Intel Xeon E5-2680 v2 (Ivy Bridge)

supports v2

cli

Doing it

qm / pct

# VM on shared storage: only RAM + device state move
qm migrate 100 pve2 --online

# VM on local disks: mirror the disks too (optionally onto another storage)
qm migrate 101 pve2 --online --with-local-disks --targetstorage local-zfs

# containers: restart mode (stop, move, start)
pct migrate 200 pve2 --restart --timeout 120

# per-VM knobs (qm set): allowed pause, bandwidth cap
qm set 100 --migrate_downtime 0.1     # seconds, default 0.1
qm set 100 --migrate_speed 0          # MiB/s, 0 = unlimited

cluster-wide migration settings

# /etc/pve/datacenter.cfg
migration: type=secure,network=10.20.0.0/24
bwlimit: migration=819200       # KiB/s cap for all migrations (~800 MiB/s)

to another cluster: qm remote-migrate

# cross-cluster (still marked experimental): the target is reached through its API
qm remote-migrate 100 4100 \
  'host=pve-b1.example.com,apitoken=PVEAPIToken=root@pam!migrate=<secret>,fingerprint=AB:CD:...' \
  --target-bridge vmbr0 --target-storage ceph-b --online

task log (abridged)

2026-10-04 10:12:01 starting migration of VM 100 to node 'pve2' (10.20.0.12)
2026-10-04 10:12:01 starting VM 100 on remote node 'pve2'
2026-10-04 10:12:03 start remote tunnel
2026-10-04 10:12:04 starting online/live migration on unix:/run/qemu-server/100.migrate
2026-10-04 10:12:04 set migration capabilities
2026-10-04 10:12:04 migration downtime limit: 100 ms
2026-10-04 10:12:05 migration active, transferred 1.1 GiB of 8.0 GiB VM-state, 1.1 GiB/s
2026-10-04 10:12:11 migration active, transferred 7.6 GiB of 8.0 GiB VM-state, 1.1 GiB/s
2026-10-04 10:12:12 average migration speed: 1.0 GiB/s - downtime 41 ms
2026-10-04 10:12:12 migration status: completed
2026-10-04 10:12:15 migration finished successfully (duration 00:00:14)

why it mattersPre-copy only converges when the link drains memory faster than the guest dirties it. Bandwidth, not RAM size, decides whether you get 40 ms or no migration at all.
why it mattersShared storage turns migration into a RAM copy. Local disks work too, but the disk copy dominates; containers always restart.
why it mattersPick a CPU type that every node in the cluster supports (e.g. x86-64-v2-AES or v3). host is only safe between identical CPUs.