Running Large Language Models (LLMs) locally using Ollama or vLLM requires bare-metal GPU performance. In a homelab, allocating a dedicated machine per GPU is inefficient. We can use Proxmox VE to host multiple VMs and pass PCIe GPUs directly through to them via VFIO, achieving near-native performance.

Host IOMMU Configuration

Before passing hardware, the host OS (Proxmox) must isolate the PCI devices. This requires enabling IOMMU in the BIOS/UEFI and kernel.

# Edit GRUB to enable IOMMU for Intel or AMD
nano /etc/default/grub
# For Intel: GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt"
# For AMD:   GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

update-grub

# Ensure necessary VFIO modules are loaded on boot
cat <<EOF >> /etc/modules
vfio
vfio_iommu_type1
vfio_pci
vfio_virqfd
EOF

update-initramfs -u -k all
reboot

Annotation: iommu=pt (passthrough) prevents Linux from translating DMA requests for devices that aren't being passed through, increasing general host performance. The VFIO modules are the kernel frameworks that handle detaching the device from host drivers and exposing it securely to userspace/QEMU.

Isolating the GPU with VFIO-PCI

We must prevent Proxmox from loading open-source drivers (nouveau, amdgpu) for the target GPU. We bind the device to the vfio-pci stub driver instead.

# Find the PCI IDs of the GPU and its Audio controller
lspci -nn | grep -i nvidia
# Output example:
# 01:00.0 VGA compatible controller [0300]: NVIDIA ... [10de:2204]
# 01:00.1 Audio device [0403]: NVIDIA ... [10de:1aef]

# Bind these specific vendor:device IDs to vfio-pci
echo "options vfio-pci ids=10de:2204,10de:1aef disable_vga=1" > /etc/modprobe.d/vfio.conf

# Blacklist host drivers
cat <<EOF > /etc/modprobe.d/blacklist.conf
blacklist radeon
blacklist nouveau
blacklist nvidia
EOF

update-initramfs -u -k all
reboot

Annotation: Passing disable_vga=1 is crucial if you are passing the primary GPU; it prevents the host kernel from attempting to initialize a framebuffer on it. Blacklisting the drivers ensures VFIO grabs the hardware first during boot sequence. You must pass both the VGA and Audio components, as they usually share an IOMMU group.

VM Configuration for PCIe Passthrough

With the host prepped, modify the VM's configuration file to attach the PCIe device.

# Edit VM config (e.g., VM ID 100)
nano /etc/pve/qemu-server/100.conf

# Add the following lines:
machine: q35
hostpci0: 0000:01:00,pcie=1,x-vga=1
# For modern NVIDIA cards, hide the hypervisor signature to prevent Code 43 errors
cpu: host,hidden=1,flags=+pcid

Annotation: The VM must use the q35 machine type, which provides native PCIe bus architecture (older i440fx only simulates PCI). pcie=1 tells QEMU to expose the device as a PCIe device. x-vga=1 designates it as the primary display output. hidden=1 strips KVM hypervisor signatures from CPUID, bypassing NVIDIA's historic blocks against consumer GPUs in VMs.