Oh how happy are we that VMware FINALLY decided to add support for NFS4.1 especially since the NFS3 client’s major problem has always been single-session connectivity. HOWEVER; going over Chris Wahl’s – extensive – coverage on this news item I did miss a key ingredient we were actually waiting for in NFS4.1: support for pNFS (parallel NFS). Chris mentions the following improvements from vSphere 5.5 to 6:
- Authentication with Kerberos
- In-band, mandatory, and stateful server side locking
- Session trunking (true NFS multipathing)
- Greater error recovery
So what does this session trunking do for us and what not? I tried to simplify the story with some graphics. A couple of sidenotes/caveats before we start:
- I am only showing front-end connectivity from the storage client (ESXi) to the server (NFS). So it’s not becasue a folder/datastore only shows on 1 server that the data is not replicated/striped in the backend to other nodes
- following this logic, two of these servers could in reality be two headers of a classical scale-up array or all headers could be independent members in a shared nothing scale-out environment
- I have only used 1 datastore per server. I know this is a caveat people will want to use to show that multiple targets will create a better balance of resources. lets keep that for another post, ok?
- I also know that pNFS is more than NAS traffic when we speak about data-plane & control-plane split but that also is for another post
Draw it on a napkin
That being said here’s my simplified view of what we have today (vSphere 5.5 and earlier)
What you see is that each target has a maximum of 1 session, no matter what the fabric design is. All other connections are merely helping with hardware failures (NIC & Switch).
What vSphere 6 is bringing to the table: adding another session to the same target. What this brings is bandwidth increase and latency decrease. Thank you VMware
What is vSphere 6 NOT bringing? True parallelism & striping. What you see below is how this would work if VMware would add support for pNFS.
My Take
Don’t get me wrong, I LOVE the fact that VMware has added support for NFS4.1. The only problem is that the scale-out architectures that have based themselves on NFS will still need that backend traffic for true parallelism and frontend striping. Maybe vSphere6.1? 😉
I don’t call myself an NFS specialist so feel free to chime in and update my knowledge! I will happily be proven wrong and add your knowledge to my drawings and notes.
[…] No support for pNFS will be available for ESXi 6.0. This has caused some confusion, best to have a look at Hans de Leenheer’s post: VSPHERE 6 NFS4.1 DOES NOT INCLUDE PARALLEL STRIPING! […]
[…] 6 includes NFSv4.1 support, it does not include pNFS! It does do multipathing or trunking (as Hans eloquently illustrates) but not parallel NFS. And vSphere is limited to a single datastore per array. And you can’t […]
I like the tone of this characterization: the 4.1 client in vSphere 6 is a strong step forward, and I’m sure that pNFS isn’t that far behind. It will be really cool to see it when it lands, because outside of scientific computing the pNFS protocol hasn’t ever really seen a lot of enterprise uptake and it will be interesting to see how that goes.
Then again, if you want to scale NFSv3 to take full advantage of wide network connectivity into a scalable storage system without needing either pNFS or vVols, there is a pretty compelling solution that’s already available.
I’ll be the first to admit that this is perhaps not an entirely unbiased opinion… 😉
Thanks for the informative post Hans!
andy.
Thx for chiming in Andy. Whilst I was writing this I somehow had CohoData in the back of my head but didn’t want it to sound like an advertisement.
As others have pointed out the lack of pNFS support is merely one of the limitations today and maybe not even the most important one (VAAI? VVols?). But it was the most undocumented one when thinking about “support for NFS4.1”.
[…] v4.1 (Cormac Hogan) vSphere 6 Features – Mark or Tag local disk as SSD disk (ESX Virtualization) vSphere 6 NFS4.1 does not include parallel striping! (Hans DeLeenheer) vSphere 6: NFS 4.1 Finally Has a Use? (Stephen Foskett) VMware SDS vision […]
[…] Hogan) vSphere 6 Features – Mark or Tag local disk as SSD disk (ESX Virtualization) vSphere 6 NFS4.1 does not include parallel striping! (Hans DeLeenheer) vSphere 6: NFS 4.1 Finally Has a Use? (Stephen Foskett) VMware SDS […]
Hello,
I’m trying to get that multipathing on my network with NFS4.1.
I have ESXI and vSphere 6.0. But what Server NAS support NFS4.1 and session trunking ?
I tried with a linux but i haven’t seen multiple sessions… (FreeNAS & Qnap aren’t running NFS4.1 …)
Thanks for your help !
pNFS separates out the file metadata from the file data access. pNFS on its own does not do striping. For striping you’d need pNFS on top of a striping file system such as Lustre or GPFS.
[…] http://hansdeleenheer.com/vsphere-6-nfs4-1-does-not-include-parallel-striping/ […]