I had a question around how Virtual SAN read IO is handled when data can be anywhere: read cache, write buffer, disks. On VMTN one of the engineers recently explained this. I figured I would create a quick diagram to illustrate it. Basically how it works is that VSAN will check the read cache, if the block that needs to be read is not available in the read cache it will check whether the block is in the write buffer or on disk. Simple right?
In the scenario I drew below two blocks needs to be read. Block 1 is actively served by ESXi-01 and Block 2 is actively served by ESXi-03. In the case of ESXi-01 the block resides in the read cache so it is read from the cache. In the case of ESXi-03 it is not in the read cache and neither in the write buffer, hence it is read from the magnetic disks. Do note that this is 1 virtual machine, so reads are being served from 2 hosts and depending who is actively serving IO for that block the block can reside on that host in the read cache. The host which is not actively serving IO for that block will also not place the block in the read cache! (Of course if the host which is actively serving IO for a block fails the other host will take over.)
I hope that helps.
Doug Baer says
Duncan, thanks for the great post. This leads me to a few questions:
1) when the block is served out of the write buffer on esxi-01, does it get copied into the read cache? I would expect so, but I don’t know for sure.
2) when the second block, served by esxi-03’s disk, is provided to esxi-01, does it go into the read cache of esxi-03 or esxi-01? Both? Neither?
Any answer on that question? 🙂
Hi Duncan !
I noticed a small error on the picture, in the text under VM it says:
“… and block 2 by esx-02”, should be esx-03
Great post anyway!
Duncan Epping says