For continued updated guidance on iSCSI and routed traffic/custom IP stacks I would like to refer you to storagehub, all iSCSI best practices can be found here.
I noticed a question today on an internal slack channel and it was about the use of custom TCP/IP stacks for iSCSI storage environments. Cormac and I updated a bunch of Core Storage whitepapers recently, and one of them was the iSCSI Best Practices white paper. It appears that this little section about routing and custom TCP/IP stacks is difficult to find, so I figured I would share it here as well. The executive summary is simple: Using custom TCP/IP stacks for iSCSI storage in vSphere is not supported.
What is nice though is that with vSphere 6.5 you can now set a gateway per vmkernel interface, anyway here’s the blurb from the paper:
As mentioned before, for vSphere hosts, the management network is on a VMkernel port and therefore uses the default VMkernel gateway. Only one VMkernel default gateway can be configured on a vSphere host per TCP/IP Stack. You can, however, add static routes from the command line or configure a gateway for each individual VMkernel port.
Setting a gateway on a per VMkernel port granular level has been introduced in vSphere 6.5 and allows for a bit more flexibility. The gateway for a VMkernel port can simply be defined using the vSphere Web Client during the creation of the VMkernel interface. It is also possible to configure it using esxcli. Note: At the time of writing the use of a custom TCP/IP Stack is not supported for iSCSI!
I hope that clarifies things, and makes this support statement easier to find.