My colleague Irfan just emailed me about a new paper he has been working on and which will be presented at the FAST 2009 conference. Irfan wrote this mind blowing paper with Ajay Gulati and Carl Waldspurger. PARDA: Proportional Allocation of Resources for Distributed Storage Access.
Some of you might recognize Carl’s name by the way cause he also wrote the famous “Memory Resource Management in VMware ESX Server” paper, which explains the effect of TPS and content based page page sharing.
I didn’t say mind blowing just to make it sound cool, this is one of those papers that will make your brain hurt… well at least for me as a Consultant it does.
A short outtake that explains what PARDA is about:
PARDA, a novel software system that enforces proportional-share fairness among distributed hosts accessing a storage array, without assuming any support from the array itself. PARDA uses latency measurements to detect overload, and adjusts issue queue lengths to provide fairness, similar to aspects of flow control in FAST TCP.
There’s one thing that stands out in my opinion after reading the paper a couple of times:
Combining a distributed flow control mechanism with a fair local scheduler allows us to provide end-to-end IO allocations to VMs. However, an interesting alternative is to apply PARDA flow control at the VM level, using per-VM latency measurements to control per-VM window sizes directly, independent of how VMs are mapped to hosts. This approach is appealing, but it also introduces new challenges that we are currently investigating. For example, per-VM allocations may be very small, requiring new techniques to support fractional window sizes, as well as efficient distributed methods to compensate for short-term burstiness.
So you can see where this might be going in the near future. Memory shares, CPU shares and Storage Shares. Combine this with OVF and vApp, which gives you the opportunity to add these details in the VM’s metadata, and you’ve got one big SLA driven virtualized environment or should I say a vCloud?!? This is not what the paper is about by the way, the paper describes what PARDA is, does and how it’s been tested… and it includes the results of course.
Irfan also wrote a short blog article on the paper you might also want to check this one, and don’t forget to add him to your bookmarks/rss reader!