I received a question last week from a customer who had implemented a stretched VSAN cluster. The Health Check after the implementation indicated that there was an “issue” with the MTU configuration. The customer had explained that he had configured an MTU of 9000 between the two data sites and an MTU of (default) 1500 between data sites and the witness.
The question of course was, why the Health Check indicated there was an issue. The problem here is that witness traffic and data in todays version of Virtual SAN use the same VMkernel interface. If the VSAN VMkernel interface on the the “data” site is configured for 9000 and one the “witness” site is configured for 1500 then there is a mismatch which causes fragmentation etc. This is what the health check calls out. VSAN (and the health check as such) expects an “end-to-end” consistently configured MTU, even in a stretched environment.
Great post, and great timing – we have just enabled our first vsan cluster, with a few more to come, and we did see the issue with jumbo frames.
We have 10 gig between the two hosts, but a normal wan connection to the witness server in another country. What is your advise on this setup
1: leave the config with 1500 MTU between all servers,
2: raise the MTU between the servers that has active disks, or
3: raise the MTU between all servers including witness server over WAN.
I’m not that much into network, so your advise on this would really be a great help for us.
In these situations your network constraints will dictate what the solution will look like, I highly recommend talking to the networking team. most likely the MTU will need to be 1500 though as that is pretty normal in a WAN environment
Great post, thanks for sharing your thoughts