A customer of mine ran a pair of NVMe-backed database VMs on Proxmox VE 8, replicating over a dedicated 25GbE link. Everything looked fine on paper — NICs rated for 25 Gbps, a clean VLAN, jumbo frames enabled — but iperf3 between the two guests topped out around 9 Gbps, and the host's vhost-* kernel threads were pegging a full core each during replication windows. That's the moment SR-IOV stops being an interesting feature in the wiki and starts being the fix you actually need.

This article walks through configuring Single Root I/O Virtualization on Proxmox VE 8.x for network cards — what it buys you over virtio-net, how to enable it correctly, where people get bitten, and how to verify it's actually doing what you think it's doing.

Problem Overview

Proxmox's default networking path for a VM is virtio-net talking to a Linux bridge on the host, backed by a vhost-net kernel thread that shuttles packets between the guest's virtqueues and the physical NIC. It's a good default. It's also not free: every packet crosses the host kernel, gets a context switch, and consumes host CPU cycles that scale with throughput and packet rate, not just bandwidth. For most workloads that overhead is invisible. For anything pushing sustained 10GbE+ traffic, doing small-packet workloads (VoIP, market data, high-transaction-rate databases), or running latency-sensitive services, it becomes the bottleneck. You end up burning host cores just moving bytes, and you introduce jitter that virtio's software queues can't fully hide.

SR-IOV solves this by letting a single physical NIC port present itself to the PCIe bus as multiple lightweight devices — Virtual Functions (VFs) — alongside the original Physical Function (PF). Each VF can be handed to a VM directly via VFIO passthrough, so the guest talks to the hardware with essentially no host kernel involvement in the data path. You trade some flexibility (no live migration without extra tooling, no software bridge features) for near-bare-metal throughput and latency.

Symptoms

Before jumping to SR-IOV, make sure you're actually looking at a virtio bottleneck and not something else. The pattern usually looks like this:

  • Host CPU usage climbs during high-throughput VM traffic, specifically in vhost-<pid> threads visible in top or htop (press H to expand threads).
  • iperf3 or similar tools show sustained throughput well below the NIC's rated speed, even with jumbo frames and multiple parallel streams.
  • mpstat -P ALL 1 on the host shows one or two cores pinned near 100% in %soft or %sys while the rest sit idle — a classic sign of single-queue virtio or interrupt affinity problems.
  • Latency-sensitive VM traffic (database replication, cluster heartbeats, storage protocols) shows periodic spikes that correlate with unrelated traffic bursts on the same bridge.
  • Inside the guest, ethtool -S <iface> shows rising tx_dropped or ring buffer overruns under load.

None of that automatically means "use SR-IOV." It means you've got a data-path bottleneck worth investigating, and SR-IOV is one of the tools — often the most effective one — for solving it when the NIC supports it.

Root Cause

The underlying issue is architectural, not a misconfiguration. Virtio-net with a Linux bridge is a software-mediated path: the guest writes to a virtqueue, vhost-net (running in the host kernel) picks it up, the bridge does its forwarding logic, and the packet eventually reaches the physical NIC driver. Every hop costs CPU cycles and adds latency. At low throughput this is noise. At high throughput it becomes the dominant cost, and because vhost-net threads are usually pinned to specific cores, you can hit a wall on CPU well before you hit a wall on network bandwidth. There's a secondary factor worth naming: without multiqueue virtio configured correctly, a single vCPU/vhost thread pair handles all traffic for an interface, so you don't get the parallelism you'd expect even on a multi-core guest. SR-IOV removes the host from the data path entirely for VF traffic. The VF is a real, if cut-down, PCIe function with its own MAC address, its own descriptor rings, and (on modern NICs) its own MSI-X interrupts. The guest's driver talks to it the same way it would talk to a physical NIC on bare metal, using VFIO to map the device's PCI BARs and interrupts directly into the guest's address space. The PF driver on the host still exists and still manages global switch settings (VLANs, spoof checking, trust mode), but it isn't touching individual packets anymore.

Diagnosis

Start by confirming the hardware and platform actually support SR-IOV before you touch any config. Check IOMMU support is active:

dmesg | grep -e DMAR -e IOMMU

You want to see lines like DMAR: IOMMU enabled and, further down, entries for each device group. If you see nothing, IOMMU either isn't enabled in the BIOS/UEFI (look for "Intel VT-d," "IOMMU," or "SR-IOV Support" depending on vendor) or the kernel command line is missing the activation flag. Next, confirm the NIC itself advertises SR-IOV capability:

lspci -vvv -s 01:00.0 | grep -i "sr-iov" -A 5

You're looking for a Single Root I/O Virtualization (SR-IOV) capability block with a TotalVFs value greater than zero. If that block isn't there, the card doesn't support SR-IOV and you're not going to get it working no matter what you do in software — this is a hardware/firmware feature, not something Proxmox can add. Check which driver the interface is using and whether it supports VF creation:

ethtool -i eno1

On Intel cards this is typically i40e (X710/XL710 family), ixgbe (X520/X540/X550), or ice (E810 family). Each of these has its own module parameter naming, which matters later when you're persisting VF count across reboots. Finally, sanity-check current IOMMU grouping before you plan which VFs go where:

for d in /sys/kernel/iommu_groups/*/devices/*; do
  n=${d#*/iommu_groups/*}; n=${n%%/*}
  echo "IOMMU Group ${d#*iommu_groups/}: $(lspci -nns ${d##*/})"
done

It's fine if a VF shares a group with its sibling VFs or its parent PF's root port — that's expected. It's a problem if it's grouped with unrelated devices you need to keep on the host, since VFIO passthrough operates at IOMMU-group granularity in the strict case (Proxmox's VF handling is more permissive than raw device passthrough here, but it's still worth checking).

Step-by-Step Solution

The overall flow is: enable IOMMU, load VFIO modules, create VFs, make VF creation persistent, then assign VFs to VMs.

1. Enable IOMMU on the kernel command line

How you do this depends on whether the node boots via systemd-boot (typical for ZFS-root installs) or GRUB (typical for ext4/LVM installs with a single boot disk). Check which one you're on:

proxmox-boot-tool status

If it reports systemd-boot, edit /etc/kernel/cmdline and append the IOMMU flag to the existing line, then refresh:

intel_iommu=on iommu=pt

proxmox-boot-tool refresh

If it's GRUB, edit /etc/default/grub, add the flag to GRUB_CMDLINE_LINUX_DEFAULT, then rebuild:

update-grub

On AMD platforms the IOMMU is on by default, but you generally still want iommu=pt (passthrough mode) for devices you're not virtualizing, to avoid unnecessary DMA translation overhead on the host's own traffic. On recent kernels (6.8+) Intel IOMMU is also enabled by default, but I still set the flag explicitly — it costs nothing and removes ambiguity when troubleshooting six months later. Reboot after this step. Confirm with the dmesg check from the Diagnosis section before continuing.

2. Load the VFIO modules

Add these to /etc/modules so they load at boot:

vfio
vfio_iommu_type1
vfio_pci

Then:

update-initramfs -u -k all
reboot

Verify after reboot:

lsmod | grep vfio

3. Create the Virtual Functions

You can create VFs on the fly for testing:

echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs

Confirm they showed up:

lspci | grep -i "virtual function"

Don't set this higher than your card's TotalVFs value, and be realistic about how many VFs you actually need — each one reserves a slice of the NIC's queue and interrupt resources, and oversubscribing a 4-queue card with sixteen VFs will just push you back into the contention you're trying to escape.

4. Make VF creation persistent across reboots

The sysfs write above doesn't survive a reboot. There are two common approaches. The one I use, because it doesn't depend on boot ordering relative to network interface naming, is a systemd unit:

[Unit]
Description=Create SR-IOV VFs on eno1
After=network-pre.target
Before=network.target

[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo 4 > /sys/class/net/eno1/device/sriov_numvfs'
RemainAfterExit=yes

[Install]
WantedBy=multi-user.target

Save that as /etc/systemd/system/sriov-eno1.service, then:

systemctl daemon-reload
systemctl enable --now sriov-eno1.service

The alternative is a driver module parameter in /etc/modprobe.d/, which works but ties VF count to module load time and varies in parameter name by driver (max_vfs for older ixgbe, unsupported on newer i40e/ice in favor of the sysfs method). I'd rather have one systemd unit per NIC than remember which driver generation supports which module option.

5. Assign static MAC addresses to VFs (recommended)

Without this, VF MAC addresses can be randomly assigned or reset on driver reload, which is annoying if you're doing DHCP reservations or firewall rules keyed on MAC:

ip link set eno1 vf 0 mac 02:00:00:AA:BB:00

Add these lines to the same systemd unit, after the VF creation line, or to a small script it calls.

6. Attach a VF to a VM

Find the PCI address of the VF you want:

lspci -d 8086: | grep -i "virtual function"

Attach it:

qm set 201 -hostpci0 0000:01:10.0

Boot the VM and confirm the interface shows up inside the guest with ip link or lspci, using whatever in-guest driver matches the VF (same family as the host driver — ixgbevf or i40evf/iavf, for instance).

Commands

# Check IOMMU is active
dmesg | grep -e DMAR -e IOMMU

# Check SR-IOV capability and max VFs
lspci -vvv -s 01:00.0 | grep -i "sr-iov" -A 5

# Check which driver an interface uses
ethtool -i eno1

# Create VFs dynamically
echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs

# List created VFs
lspci | grep -i "virtual function"

# Show VF status, MAC, VLAN, spoof check from the PF side
ip link show eno1

# Assign a static MAC to a VF
ip link set eno1 vf 0 mac 02:00:00:AA:BB:00

# Assign a VLAN to a VF (switch-independent tagging at the PF level)
ip link set eno1 vf 0 vlan 100

# Attach a VF to a VM by PCI address
qm set 201 -hostpci0 0000:01:10.0

# Remove all VFs (must be done before changing sriov_numvfs down or up again)
echo 0 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs

# Verify VFIO modules are loaded
lsmod | grep vfio

# Rebuild initramfs after module changes
update-initramfs -u -k all

# Refresh systemd-boot kernel cmdline
proxmox-boot-tool refresh

# Rebuild GRUB config after cmdline changes
update-grub

Configuration Examples

/etc/kernel/cmdline (systemd-boot systems, Intel platform):

root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt

/etc/default/grub snippet (GRUB systems, AMD platform):

GRUB_CMDLINE_LINUX_DEFAULT="quiet iommu=pt"

/etc/modules:

vfio
vfio_iommu_type1
vfio_pci

/etc/systemd/system/sriov-eno1.service, creating 4 VFs with fixed MACs and one VLAN-tagged VF for a management network:

[Unit]
Description=Create SR-IOV VFs on eno1
After=network-pre.target
Before=network.target

[Service]
Type=oneshot
RemainAfterExit=yes
ExecStart=/bin/sh -c '\
  echo 4 > /sys/class/net/eno1/device/sriov_numvfs; \
  sleep 1; \
  ip link set eno1 vf 0 mac 02:00:00:AA:BB:00; \
  ip link set eno1 vf 1 mac 02:00:00:AA:BB:01; \
  ip link set eno1 vf 2 mac 02:00:00:AA:BB:02 vlan 100; \
  ip link set eno1 vf 3 mac 02:00:00:AA:BB:03'

[Install]
WantedBy=multi-user.target

VM network device config for a VF-backed interface, shown as it appears in /etc/pve/qemu-server/201.conf:

hostpci0: 0000:01:10.0,pcie=1
net0: virtio=BC:24:11:00:00:01,bridge=vmbr0

Note that net0 here is a separate, ordinary virtio interface for management access — keep a virtio/bridge NIC on VMs using SR-IOV VFs for anything you need to reach even if the VF path has an issue. It's saved me more than once.

Common Mistakes

The most common failure I see is enabling IOMMU in the kernel command line but forgetting it also has to be turned on in the BIOS/UEFI. Software flags can't override a platform-level setting that's off at the firmware level; you'll get an IOMMU that reports as active but with no device groups, or dmesg errors about DMAR faults. Second most common: creating VFs manually for a test, confirming everything works, and then losing them on the next reboot because nothing persisted the sriov_numvfs write. This one is sneaky because it works fine right up until the next maintenance window. People also assume SR-IOV-attached VMs can live migrate like any other VM. They can't, not without extra work — VFIO passthrough devices are tied to the physical hardware on that specific node. If you need HA or live migration on a VM using a VF, you either need a bonding/failover setup inside the guest (a virtio NIC plus the VF, with the VF preferred and the virtio one as fallback), or you accept that VM lives on that node until you manually migrate with downtime. Another one: forgetting that VLAN tagging on a VF is often enforced at the PF/switch level, not left to the guest. If you tag VLAN 100 on the VF via ip link set ... vf 0 vlan 100 and then also configure an 802.1Q subinterface inside the guest, you'll double-tag and traffic silently disappears. Pick one place to do VLAN tagging and stick with it. Last one worth calling out: mismatched MTU between the VF and the rest of the path. SR-IOV doesn't magically fix an MTU mismatch upstream — if your switch port or bridge is set to 1500 and you're pushing jumbo frames from the VF, you'll get fragmentation or drops that look like a driver problem but aren't.

Best Practices

Size your VF count to actual need, not to the card's maximum. A 4-port 25GbE NIC with TotalVFs of 64 doesn't mean you should run 64 VMs off it; queue depth and interrupt resources are finite and shared. Keep a management-plane virtio NIC on any VM using an SR-IOV VF, on a separate bridge, so you always have a fallback path if the VF or its PF driver has an issue. Document the VF-to-VM mapping somewhere outside Proxmox itself — a spreadsheet, a wiki page, whatever you already use for IP allocation. PCI addresses for VFs aren't self-explanatory six months later when you're trying to figure out why VM 214 won't boot after a NIC firmware update. Pin static MACs on VFs from day one rather than retrofitting it after you've built firewall rules or DHCP reservations around whatever the driver assigned at first boot. Keep NIC firmware and the host driver in sync with what Proxmox's kernel actually ships. SR-IOV VF behavior, especially around VLAN offload and trust mode, has changed between driver generations on both Intel and Mellanox cards, and an out-of-date firmware is a real source of "it works on my other node" bugs. Test failover behavior before you need it. If your plan for migration or hardware failure is a guest-side NIC bond between a VF and a virtio interface, actually pull the VF out from under a running VM in a maintenance window and confirm the guest OS handles the failover the way you expect. Don't find out during a real outage.

FAQ

Can I live migrate a VM that has an SR-IOV VF attached?
Not directly. VFIO passthrough ties the VM to the physical VF on that host. Some setups work around this with a guest-side network bond (VF plus virtio, VF preferred), detaching the VF before migration and reattaching after, but there's no built-in live migration path for the VF itself.

Does SR-IOV work with Open vSwitch instead of a Linux bridge?
The VF traffic path bypasses the bridge entirely, whether it's OVS or a Linux bridge, since the guest talks straight to the hardware. OVS with hardware offload is a separate feature (OVS-DPDK or OVS with SR-IOV offload) and a different configuration path than what's covered here.

Which NICs have solid SR-IOV support on Proxmox VE 8?
Intel's X520/X540/X550 (ixgbe), X710/XL710 (i40e), and E810 (ice) families are well-trodden. Mellanox/NVIDIA ConnectX-4 and newer also work well and support additional features like VF-LAG. Consumer or lower-end NICs frequently lack SR-IOV entirely — check TotalVFs before assuming it'll work.

Do I need a managed switch for VLAN tagging on VFs?
No, VLAN tagging can be enforced at the PF via ip link set <pf> vf <n> vlan <id>, independent of switch configuration, though your upstream switch port still needs to be trunked correctly for the VLAN to actually reach anywhere.

Is the performance gain worth the added complexity for a homelab?
For most homelab traffic patterns, no — virtio-net is plenty and far easier to manage, especially since you lose live migration flexibility. It's worth it if you're specifically testing NFV-style workloads, doing sustained high-throughput storage replication, or running latency-sensitive services where you want production-representative numbers.

Conclusion

SR-IOV isn't something to reach for by default. Virtio-net with a Linux bridge covers the overwhelming majority of Proxmox workloads without the operational cost of losing live migration and taking on hardware-specific configuration. But when you've actually diagnosed a host-side data-path bottleneck — vhost threads pinned at 100%, throughput capped well under the NIC's rating, latency that tracks unrelated traffic on the same bridge — SR-IOV is the right tool, and it's a well-supported one on current Proxmox VE releases with the right hardware underneath it. Get the IOMMU enablement right at both the firmware and kernel level, persist VF creation with a systemd unit instead of a one-off sysfs write, and keep a fallback network path on any VM that depends on a VF. Do those three things and SR-IOV is boring in the good sense: it works, it's fast, and you stop thinking about it until the next hardware refresh.