Configuring PCIe GPU Passthrough in Proxmox for Local LLMs

Running LLMs locally with tools like Ollama or vLLM requires real GPU performance. But dedicating an entire physical machine to a single GPU is wasteful in a homelab environment. Proxmox VE lets you run multiple VMs on a single host and pass PCIe GPUs straight to your guest VMs using VFIO, giving you near-native performance.

Host IOMMU Configuration

Before passing hardware to a VM, Proxmox must isolate the PCI devices. You need to enable IOMMU in your BIOS/UEFI and configure the kernel.

# Edit GRUB to enable IOMMU for Intel or AMD
nano /etc/default/grub
# For Intel: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
# For AMD: GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"
update-grub
# Ensure VFIO modules load on boot
cat <<EOF >> /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
EOF
update-initramfs -u -k all
reboot

Note: Setting iommu=pt (passthrough) prevents Linux from translating DMA requests for host devices you are not passing through, which improves host performance. The VFIO modules handle detaching the card from host drivers and safely exposing it to QEMU.

Isolating the GPU with VFIO-PCI

Next, prevent Proxmox from loading default open-source drivers like nouveau or amdgpu for the GPU. Bind the hardware to the vfio-pci driver instead.

# Find PCI IDs for the GPU and its audio controller
lspci -nn | grep -i nvidia
# Example output:
# 01:00.0 VGA compatible controller [0300]: NVIDIA ... [10de:2204]
# 01:00.1 Audio device [0403]: NVIDIA ... [10de:1aef]
# Bind vendor:device IDs to vfio-pci
echo "options vfio-pci ids=10de:2204,10de:1aef disable_vga=1" > /etc/modprobe.d/vfio.conf
# Blacklist host drivers
cat <<EOF > /etc/modprobe.d/blacklist.conf
blacklist radeon
blacklist nouveau
blacklist nvidia
EOF
update-initramfs -u -k all
reboot

Note: Adding disable_vga=1 is crucial if you are passing the primary GPU. It prevents the host kernel from configuring a display framebuffer on it. Blacklisting host drivers ensures VFIO claims the card first during boot. Make sure to pass both the VGA and Audio functions together, as they usually share an IOMMU group.

VM Configuration for PCIe Passthrough

With the host configured, update the VM config file to attach the GPU.

# Edit VM config (e.g., VM ID 100)
nano /etc/pve/qemu-server/100.conf
# Add the following lines:
machine: q35
hostpci0: 0000:01:00,pcie=1,x-vga=1
# For modern NVIDIA GPUs, hide hypervisor flags to prevent Code 43 errors
cpu: host,hidden=1,flags=+pcid

Note: The VM requires the q35 machine type, which provides a native PCIe bus (i440fx only emulates legacy PCI). pcie=1 exposes the card as a PCIe device, while x-vga=1 marks it as the primary display. Setting hidden=1 hides KVM hypervisor signatures from CPUID to bypass NVIDIA driver Code 43 errors.