SDN zones & VNets

Bridges and VLANs in /etc/network/interfaces are configured one node at a time. Proxmox SDN (installed and enabled by default since PVE 8.1) defines virtual networks once for the whole cluster. A zone chooses how traffic crosses between nodes, a VNet is one network in that zone that guests plug into, and a subnet adds IP addressing, a gateway and optionally DHCP. Pick a zone type to see how VM 100 on pve1 reaches VM 200 on pve2.

Two VMs in vnet1 (10.0.20.0/24) on different nodes

VNet MTU 1450underlay 1500 − 50 bytes of headers

Layer 2 over layer 3: each Ethernet frame is wrapped in UDP (port 4789) and sent to the other nodes' IPs from the zone's peer list. The underlay only needs IP connectivity, no VLANs. Flooding (broadcast, unknown unicast) is copied to every peer.

underlay MTU1500

The frame on the wire between pve1 and pve2

+50 bytes vs. the VM's frame

outer Eth

14 B

outer IPv4

20 B

UDP

8 B

VXLAN

8 B

inner Eth

14 B

IP + TCP + data

≤ MTU

Hover a header to see what it carries.

VXLAN header, RFC 7348 (8 bytes = 2 × 32 bits)

00001000reserved (24)000000011000011010100000rsvd

flags · I bit = VNI validVNI (24 bits): 16,777,216 networks vs 4094 VLANs

VNet tag (VNI)0x0186a0

14 + 20 + 8 + 8 = 50 bytes of new headers. The outer IP packet must fit the underlay MTU, so the VNet's MTU (and the guests') must be 50 bytes smaller, or the underlay MTU 50 bytes larger. Otherwise large packets fail while ping and SSH still work.

vxlan zone · /etc/pve/sdn/*.cfg

# /etc/pve/sdn/zones.cfg
vxlan: overlay
        peers 10.10.5.11,10.10.5.12,10.10.5.13
        mtu 1450
        ipam pve

# /etc/pve/sdn/vnets.cfg
vnet: vnet1
        zone overlay
        tag 100000

# generated on pve1 → /etc/network/interfaces.d/sdn
auto vxlan_vnet1
iface vxlan_vnet1
        vxlan-id 100000
        vxlan_remoteip 10.10.5.12
        vxlan_remoteip 10.10.5.13
        mtu 1450

auto vnet1
iface vnet1
        bridge_ports vxlan_vnet1
        bridge_stp off
        bridge_fd 0
        mtu 1450

choose by what your underlay can do

Zones at a glance

The key question is what the physical network gives you. If the switches carry VLANs, VLAN zones are the simplest. If you only have routed IP between nodes (another rack, another site, or a hoster), use VXLAN or EVPN.

	Simple	VLAN	QinQ	VXLAN	EVPN
Crosses nodes	no (routed/SNAT only)	yes, via switch	yes, via switch	yes, UDP 4789	yes, UDP 4789
Underlay needs	nothing	VLAN trunk	S-VLAN trunk	IP reachability	IP reachability
Isolation IDs	per node	4094	4094 × 4094	16M VNIs	16M VNIs + VRFs
MTU cost	0	0	4 bytes	50 bytes	50 bytes
Gateway	on each node	external router	external router	external router	anycast on every node
MAC learning	local bridge	switch	switch	flood to peers	BGP announcements

pending → running

Apply: how config becomes interfaces

SDN does not replace ifupdown2. It generates config for it. That's why you can debug any SDN network with the usual tools (ip -d link, bridge fdb, tcpdump -i vnet1).

From a click to a running VNet

/etc/pve/sdn/{zones,vnets,subnets,controllers}.cfg

Creating a zone, VNet or subnet in the GUI (or with pvesh/API) only writes these files in pmxcfs. They are replicated to all nodes at once, but nothing on the network changes yet: the GUI shows the objects as pending.

pve1

pending

pve2

pending

pve3

pending

who hands out the addresses

IPAM, DHCP and DNS

Subnets can be tracked by an IPAM plugin. When a guest gets a NIC on a VNet with a subnet, Proxmox can reserve an IP for its MAC and, in zones with DHCP enabled, serve it.

	What it does	Notes
pve IPAM	built-in, stored in pmxcfs	default; shown in the GUI's IPAM view
NetBox	external IPAM via its API	NetBox stays the source of truth
phpIPAM	external IPAM via its API	section/subnet must already exist
DHCP	dnsmasq instance per zone	Simple zones; install dnsmasq first
DNS	PowerDNS plugin	creates A/PTR records for guests

using it

# a VNet is used like any bridge
qm set 100 --net0 virtio,bridge=vnet1

# apply pending SDN changes from the CLI
pvesh set /cluster/sdn

# inspect what got generated
cat /etc/network/interfaces.d/sdn
vtysh -c "show bgp l2vpn evpn summary"     # EVPN zones

Tech-preview edges

The IPAM/DHCP integration is still marked tech preview in the docs. DHCP leases come from IPAM mappings, not from a dynamic pool, so a guest NIC created outside Proxmox's knowledge gets nothing.

controllers, VRFs, exit nodes

EVPN in one paragraph

An evpn controller makes Proxmox write an FRR config on every node: BGP sessions to the peers, with the l2vpn evpn address family. Each EVPN zone is a VRF with its own L3 VNI (vrf-vxlan), so subnets of the zone can route to each other directly on the local node, and tenants in different zones stay separated. To leave the overlay, traffic goes to the exit nodes, which announce a default route into the zone and route (optionally SNAT) toward the physical network. Optionally, a bgp controller peers with your real routers.

new in 9.x

PVE 9: fabrics for the underlay

VXLAN and EVPN need IP routes between all nodes' VTEP addresses. Before 9.0 you configured that underlay by hand. Proxmox VE 9.0 added SDN fabrics: you list the nodes and their interfaces, and Proxmox configures OpenFabric or OSPF in FRR so every node learns routes to every other node's loopback. This works well with full-mesh links that have no switch (for example a 3-node Ceph mesh). An EVPN controller can then use the fabric as its underlay. PVE 9.2 added WireGuard and BGP as further fabric protocols.

why it mattersZone = how VNet traffic crosses nodes. Simple stays on one node, VLAN and QinQ rely on switch tags, VXLAN and EVPN wrap frames in UDP 4789 and only need routed IP between nodes.
why it mattersEvery encapsulation costs MTU: 4 bytes for QinQ, 50 bytes for VXLAN/EVPN. Lower the VNet MTU or raise the underlay MTU, or large packets will be dropped while small ones work.
why it mattersSDN edits are pending until Apply. Each node then writes its own /etc/network/interfaces.d/sdn (and FRR config) and runs ifreload, so you can debug VNets with the usual Linux tools.