thinking sysadmin

Scott Lowe recently linked to a VMware KB article entitled Storing swap files on VMFS when running virtual machines from NFS. The article (from 3/31/2008) is perhaps the latest word from VMware in the frustrating back-and-forth on whether placing an ESX VM’s swap on NFS is acceptable or not.

The KB entry itself isn’t directly about (not) placing VM swap on NFS - it details how to specify the location of swap in a VM’s config file. Regarding the question of swap location, the article (weakly) recommends against placing swap on NFS:

This articles describes changing the default location for your virtual machines [sic] swap files. You may need to use this if you are running virtual machines from NFS storage.
Note: VMware recommends storing your swap on a VMFS3 volume, when running virtual machines on NFS storage.

(Emphasis mine.) The article provides none of the reasoning behind this recommendation, and a search of the VMware Knowledge Base for “nfs swap” only returns this article. There are quite a few mentions of not putting swap on NFS in the VMware Communities forums, citing performance reasons, but, as far as I could find, no support for the performance issue assertion.

Personally, I’m not buying the performance claims without substantiation - NFS and iSCSI to the same filer should be nearly equally performant. I could imagine a difference in locking behavior, but I’m having trouble imagining a scenario where lock contention with a VM swap file would become an issue.

On the other side of the issue, I’ve been hearing from a few folks - mostly NetApp employees or users - that placing swap on NFS is okay. On the viops site, Steve Chambers writes in a preview of VMworld TA 2784 (which I did not attend) that “there is no need to place swap space on non IP storage.” IP storage could mean iSCSI here, of course, but in the VM /ETC write up of TA 2784, Rich Brambley’s notes state: “Do not place [swap] space on other storage – do not place the vswap on VMFS. Follow best practices and keep the swap on the NFS storage.”

So, who’s right here? VMware or NetApp? I suppose the conservative thing is to keep the swap on VMFS until VMware tells us otherwise, with the only negatives being the increased operational complexity and reduced storage utilization - that’s what I’m doing. I can’t help but wonder if this is like VMware’s Linux timekeeping recommendations, though - conflicting information subject to change - and hopefully clarification - at a later date.

Update, 11/24/2008: Keeping ESX swap on NFS with the VMhome directory is considered the current best practice.