Every hypervisor eventually runs into the same wall: virtio-net is fast enough for most workloads, until it isn't. I've watched a perfectly healthy-looking Proxmox VE node choke a single VM down to 2-3 Gbps on a 10GbE link, not because the NIC was slow, but because one vhost- kernel thread was pinned at 100% on a single core while seven others sat idle. That's the point where you stop tuning virtio and start looking at SR-IOV.
Single Root I/O Virtualization lets a physical NIC present itself to the kernel as multiple lightweight PCIe devices, called virtual functions, that you can hand directly to a VM the same way you'd pass through a GPU. The guest talks almost straight to the hardware. No vhost thread, no software bridge, no per-packet copy through the host kernel. On PVE 9.x this works well with Intel's ixgbe/i40e/ice-based cards and a growing list of Mellanox and Broadcom parts, and it's one of the more reliable ways to get near bare-metal throughput and latency out of a virtualized network path.
Problem Overview
Standard virtio-net networking on Proxmox routes every packet through the host's network stack: the vhost-net kernel thread services virtqueue notifications, the Linux bridge (or OVS) does MAC learning and forwarding, and only then does the frame hit the physical NIC's queue. That software path is what gives you live migration, hot-add NICs, and painless snapshots. It's also where the CPU cycles go.
For most workloads this is a non-issue. A file server, a web app, a database replicating over a 1GbE link — none of that cares whether there's a vhost thread in the way. But once you're pushing sustained 10GbE+ traffic, running latency-sensitive workloads like NVMe-oF initiators, packet capture appliances, or anything doing line-rate small-packet forwarding, that software path becomes the bottleneck, and it becomes one long before the physical NIC does.
SR-IOV solves this by moving the data plane out of the host kernel entirely. The VF is a real PCIe function with its own MAC address, its own hardware queues, and its own slice of the NIC's internal switch. The tradeoff is real too: you give up live migration for any VM holding a VF (short of manual bonding tricks with a fallback virtio NIC), and you're now managing MAC addresses and VF counts as infrastructure, not something the GUI does for you.
The workloads where this actually matters tend to look similar across environments. A homelab user running a pfSense VM that needs to route real traffic between VLANs at close to line rate. An MSP hosting a client's NVMe-oF target where every microsecond of added latency shows up in storage benchmarks. A telco or NFV shop running a virtualized packet processing appliance that needs to inspect every frame at 10Gbps without dropping any. In all three cases, the difference between virtio and SR-IOV isn't marginal — it's the difference between 3 Gbps and 9.4 Gbps on the same cable, and between single-digit microsecond latency and something an order of magnitude worse.
Symptoms
You'll usually notice this the same way I did — via a slow support ticket rather than a proactive check. A few things tend to show up together:
- iperf3 throughput plateaus well below the link's rated speed, often somewhere between 2 and 4 Gbps on a 10GbE NIC, no matter how many parallel streams you throw at it.
- top or htop on the Proxmox host shows a vhost-<pid> process pinned near 100% CPU on exactly one core, while the rest of the host sits comfortably under 30%.
- Latency-sensitive traffic (VoIP, market data feeds, cluster heartbeats) shows jitter spikes that correlate with unrelated I/O load on the same host — because everything is contending for the same host networking stack.
- Guest-side ethtool -S shows dropped or missed packets under load that the physical NIC's own counters don't report as errors, which tells you the bottleneck is in software, not on the wire.
None of these are dramatic. That's what makes them easy to write off as "the VM just needs more vCPUs," which doesn't actually fix anything — the vhost thread bottleneck is per-queue, not something you resolve by throwing more cores at the guest.
Root Cause
The root cause is architectural, not a misconfiguration. virtio-net was designed for flexibility, not for saturating hardware. Every frame leaving a VM crosses the virtqueue into the vhost-net kernel thread, which does the actual send/receive work on behalf of QEMU. That thread is, by default, single-threaded per queue pair unless you've configured multiqueue virtio-net and pinned the guest's IRQs to match — and even then, you're still burning host CPU cycles on packet copies and bridge lookups that a physical NIC's ASIC could do at wire speed.
Add a Linux bridge or Open vSwitch in the path and you've got MAC table lookups and, depending on configuration, VLAN tag handling happening in software on every single frame. None of this shows up as an "error" anywhere. The VM sees a working NIC. The host sees normal load. What you're actually looking at is a CPU-bound software switch masquerading as a 10GbE port.
I'd never blame the NIC vendor's driver here — I've seen the exact same ceiling on Intel, Mellanox, and Broadcom cards. The bottleneck is the host-side emulation path, not the silicon.
Diagnosis
Before you touch any config, confirm the NIC actually supports SR-IOV and that you're hitting a software ceiling rather than something else entirely.
Start with the capability check:
lspci -vvv -s 01:00.0 | grep -i "single root"If you see SR-IOV listed under the device's capabilities, you're clear to proceed. If it's missing, the card doesn't support it — don't waste time on kernel cmdline changes.
Next, confirm where the CPU time is actually going during a load test. Run an iperf3 test from the guest and watch the host in parallel:
mpstat -P ALL 1If one core sits near 100% while the interface stays well under its rated bandwidth, that's the vhost thread doing exactly what it's built to do — working as hard as it can on a single-threaded path. Cross-reference with:
ps -eo pid,pcpu,comm | grep vhostto find which vhost thread is pegged and correlate it back to the affected VM. At this point you've confirmed the bottleneck is host-side packet processing, not the guest's network stack, not the physical switch, and not the NIC's own throughput ceiling.
It's also worth ruling out the cheaper fixes first. Check whether virtio multiqueue is even enabled on the affected VM's NIC — a single-queue virtio-net device can only ever use one vhost thread no matter how many vCPUs the guest has. If queues isn't set on the net0 line in the VM config, that alone might explain a chunk of what you're seeing, and it's a five-minute fix compared to a full SR-IOV rollout. Only move on to SR-IOV once multiqueue and IRQ pinning have been tried and the ceiling is still there.
If you want a clean before/after number to justify the work to whoever signs off on the change window, capture a baseline with:
iperf3 -c <target> -P 4 -t 30and save the output. It's the easiest way to prove the migration actually helped, rather than relying on "it feels faster."
Step-by-Step Solution
This assumes an Intel NIC using the ixgbe or i40e driver on PVE 9.x with IOMMU support in the CPU (VT-d on Intel, AMD-Vi on AMD).
1. Enable IOMMU in firmware and kernel
Go into the server's BIOS/UEFI and enable VT-d (Intel) or the IOMMU option (AMD — usually on by default). Then, on the Proxmox host, edit the kernel command line. If the host boots via systemd-boot (the default for ZFS-on-root installs), edit /etc/kernel/cmdline. If it boots via GRUB, edit /etc/default/grub instead.
Add intel_iommu=on iommu=pt for Intel, or amd_iommu=on iommu=pt for AMD. The iommu=pt flag puts devices not being passed through into passthrough mode too, which avoids unnecessary translation overhead for the rest of the system.
Apply the change and reboot:
# systemd-boot systems
proxmox-boot-tool refresh
# GRUB systems
update-grub2. Load the VFIO modules
Add the following to /etc/modules:
vfio
vfio_iommu_type1
vfio_pciThen regenerate the initramfs so the modules load early enough in boot:
update-initramfs -u -k allReboot, then confirm IOMMU actually came up:
dmesg | grep -e DMAR -e IOMMUYou're looking for a line confirming IOMMU or Directed I/O is enabled. If it isn't there, double check the BIOS setting before going any further — nothing past this point will work without it.
3. Create the virtual functions
Find the physical NIC's PCI address with lspci -nn, then create VFs at runtime through sysfs:
echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfsFour is a reasonable starting point for a dual-port 10GbE card if you're splitting it across a handful of VMs — check your NIC's datasheet for the actual maximum, since it varies by model and firmware.
This step doesn't survive a reboot on its own, which is the single most common thing people forget. You need a systemd service (shown below) that recreates the VFs and reassigns MAC addresses on every boot, and you need the physical NIC's interface set to autostart in the Proxmox GUI under System > Network — otherwise VF creation fails silently before the interface is even up.
4. Assign a stable MAC address to each VF
ip link set eth0 vf 0 mac 02:00:00:00:00:01Without this, VFs get a randomly generated MAC on every boot, which plays badly with DHCP reservations and anything doing MAC-based access control upstream.
5. Attach the VF to a VM
Once the VF shows up as its own PCI device (check with lspci -nn again — it'll have a different function number than the physical function), pass it through like any other PCI device:
qm set 105 -hostpci0 01:10.0,pcie=1Boot the VM and confirm the guest sees a real NIC:
ethtool -i eth0You should see the actual vendor driver (ixgbevf, for example), not virtio_net. That's your confirmation the traffic path is now hardware, not emulated.
Commands
| Purpose | Command |
|---|---|
| Check SR-IOV capability | lspci -vvv -s <pci-addr> | grep -i "single root" |
| List PCI devices with IDs | lspci -nn |
| Check driver in use | lspci -k -s <pci-addr> |
| Verify IOMMU enabled | dmesg | grep -e DMAR -e IOMMU |
| Create VFs at runtime | echo N > /sys/bus/pci/devices/<addr>/sriov_numvfs |
| Check current VF count | cat /sys/bus/pci/devices/<addr>/sriov_numvfs |
| Assign VF MAC | ip link set <iface> vf <n> mac <mac> |
| Attach VF to VM | qm set <vmid> -hostpci0 <pci-addr>,pcie=1 |
| Confirm host CPU bottleneck | mpstat -P ALL 1 |
| Find pegged vhost thread | ps -eo pid,pcpu,comm | grep vhost |
Configuration Examples
/etc/kernel/cmdline (systemd-boot, Intel host):
root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt/etc/modules:
vfio
vfio_iommu_type1
vfio_pciA systemd unit to recreate VFs and assign MACs on every boot, so you're not stuck redoing this by hand after every kernel update:
[Unit]
Description=Create SR-IOV VFs on eth0
After=network.target
[Service]
Type=oneshot
ExecStart=/bin/sh -c 'echo 4 > /sys/bus/pci/devices/0000:01:00.0/sriov_numvfs'
ExecStart=/sbin/ip link set eth0 vf 0 mac 02:00:00:00:00:01
ExecStart=/sbin/ip link set eth0 vf 1 mac 02:00:00:00:00:02
RemainAfterExit=true
[Install]
WantedBy=multi-user.targetSave this as /etc/systemd/system/sriov-vfs.service and enable it with systemctl enable sriov-vfs.service.
VM config snippet (/etc/pve/qemu-server/105.conf):
hostpci0: 0000:01:10.0,pcie=1
net0: virtio=BC:24:11:AA:BB:CC,bridge=vmbr0Notice the VM still has a standard virtio NIC on net0 alongside the passed-through VF. That's deliberate — see Best Practices below.
Common Mistakes
Forgetting to set autostart on the physical NIC is the single most common failure I see. Without it, the interface isn't up when the systemd unit tries to create VFs at boot, and you end up with a VM that fails to start because its hostpci device doesn't exist.
Assuming SR-IOV VMs can live-migrate like any other guest is another one. They can't — not without a bonded fallback NIC and manual intervention, because the VF is tied to physical hardware on one specific node. If you need HA or migration for that workload, SR-IOV probably isn't the right tool, and you're better off tuning virtio multiqueue instead.
People also tend to over-allocate VFs relative to what the NIC's firmware and driver actually support well. Just because a card lets you create 32 VFs doesn't mean performance holds up at that count — check the datasheet, and test at your target count before committing to a design.
Finally, skipping the MAC assignment step causes more support tickets than anything else on this list. A VF with a random MAC on every boot breaks DHCP reservations, breaks 802.1X if you're using it, and makes troubleshooting a nightmare three months later when nobody remembers the interface's MAC changes on every reboot.
One more that catches people out later rather than immediately: firmware updates to the physical NIC sometimes reset the max VF count or reorder PCI function numbers on the next boot. If a hostpci device suddenly points at the wrong VF after a firmware or driver update, check lspci -nn again before assuming the VM config got corrupted — it's usually the addressing that shifted, not the config.
Best Practices
Keep a standard virtio NIC alongside the SR-IOV VF on any VM where you can afford it, even if you're not actively bonding them. It gives you an emergency management path if the VF driver misbehaves inside the guest, and it means you're not locked out of the VM over SSH just because a PCI passthrough device hung.
Document your MAC-to-VF mapping somewhere outside of the systemd unit itself — a comment in the file is fine, but if that host goes down and someone else has to rebuild it, they need to know why VF 3 has that specific MAC and not a different one.
Monitor the physical NIC's own counters, not just the guest's. ethtool -S on the host-side physical function will show you VF-level statistics on most Intel cards, and that's a much better early warning than waiting for a user to complain about packet loss.
Test your target workload at the actual VF count you plan to run in production before rolling it out. Four VFs performing well doesn't guarantee eight will, particularly on cards with a smaller internal switch fabric.
FAQ
Does SR-IOV work with Open vSwitch on Proxmox?
No — a VF bypasses OVS entirely by design, since the whole point is skipping the host's software switch. If you need OVS features like VLAN tagging or flow rules, apply them at the physical switch or NIC firmware level instead.
Can I use SR-IOV with a virtio-net fallback for live migration?
You can bond a VF and a virtio interface inside the guest so the VM keeps working during migration on the virtio path, but this requires guest-side configuration and isn't something Proxmox manages automatically.
Why did my VFs disappear after a kernel update?
VF creation via sysfs doesn't persist across reboots on its own. If you didn't set up the systemd service described above, every reboot — kernel update or not — wipes them.
Does this work on consumer motherboards?
Only if the board's firmware actually exposes IOMMU groups cleanly and the NIC supports SR-IOV. Plenty of consumer boards have buggy or incomplete ACS support, which shows up as devices sharing IOMMU groups that shouldn't.
Conclusion
SR-IOV isn't something to reach for by default — for the vast majority of VMs on a Proxmox cluster, virtio-net with a properly tuned bridge is the right answer, and it comes with live migration and HA for free. But when you've genuinely hit the ceiling of software-emulated networking, and you've confirmed it with mpstat rather than guessed at it, passing a real virtual function into the guest is one of the few options that actually removes the bottleneck instead of just moving it somewhere else. Set up the persistence pieces properly the first time, and it's a stable, low-maintenance part of the stack from then on.