Configure Passthrough-VMDirectPath in VMWare ESXi for RAID Cards, HBAs, and USB Drives
Oftentimes people talk about passing a RAID controller/ HBA/ USB drive through to a guest OS in VMware ESXi (this guide is based on 4.1 and works for ESX too) using VMDirectPath I/O or disk through disk in Microsoft Hyper-V. This is especially useful when one either needs direct RAID controller/ HBA access for an operating system which is recommended for OpenSolaris/ FreeBSD solutions or USB access for an OS installation. It is a fairly simple process, so I figured I would provide a quick guide. Before embarking on this process, please note that the server will need to be restarted mid-process so this is not something that should be done on an ESX host with critical virtual machines running on the host.
- CPU(s): Intel Xeon X3440
- Motherboard: Supermicro X8SI6-F
- Memory: 2x 4GB kits of Kingston ECC 1333MHz DDR3 KVR1333D3E9SK2/4G (Unbuffered)
- Case: Norco RPC-4220
- Drives: 8x Hitachi 7,200rpm 2TB, 2x Western Digital Green 2TB, 6x Seagate 1.5TB 7,200rpm
- OS(es): VMWare ESXi 4.1
First, make sure that your CPU and motherboard support Intel VT-d or AMD IOMMU. Once you are sure that the specs support VMDirectPath, you should double-check that the features are enabled in the BIOS. If the system is remote, you can still do this with IPMI 2.0 and KVM-over-IP. Once this check is complete, let the server boot.
Next log into the vSphere client and connect to your server. Select the appropriate server and go to the Configuration Tab.
From there, go to Advanced Settings where passthrough can be configured by clicking the button labeled “Configure Passthrough”
A window will then pop up asking to select a device to pass through (note actually associating the passthrough device with a VM occurs later.) Make sure that you select the correct device, a lot of USB controllers for example look very similar on this screen, so you may need to download and view the motherboard manual when figuring out what to pass through. Once the correct check box has been selected, click OK.
At this point, one needs to do a step I forgot my first time doing passthrough of a controller… and that is actually rebooting the ESXi host.
After the reboot occurs, the next important step is to make sure that the virtual machine where the PCIe or USB device is being passed through to is in an off state. This is important for machines where the target VM automatically starts up upon server boot.
Once you are sure the VM is off, go to the Summary tab -> Edit Settings and then add a PCI Device followed by clicking next. From the installation wizard that opens, pick the PCI/ PCIe device from the drop-down menu followed by clicking Next and Finish. After this is complete, wait a few seconds while the VM configuration is completed and then you can start the VM. Personally, I prefer doing the above configuration before installing an operating system to ensure the highest chance of working drivers being installed in the initial installation.
A word of caution: not every device will work with this feature, but I have had nothing but luck installing RAID controllers in this manner. Another note is that installing a RAID card as a passthrough device will mean that the attached disks are not available for other VMs. So best practice is to make sure that every drive attached to the RAID card or HBA is destined to be attached to the VM in question. I often use either onboard SATA or a dedicated controller to host ESXi and drives for other VMs while passing through a RAID card (often with a HP SAS Expander) to VMs.