• Skip to primary navigation
  • Skip to main content
  • Skip to primary sidebar

Yellow Bricks

by Duncan Epping

  • Home
  • ESXTOP
  • Stickers/Shirts
  • Privacy Policy
  • About
  • Show Search
Hide Search

How does Mem.MinFreePct work with vSphere 5.0 and up?

Duncan Epping · Jun 14, 2013 ·

With vSphere 5.0 VMware changed the way Mem.MinFreePct worked. I had briefly explained Mem.MinFreePct in a blog post a long time ago. Basically Mem.MinFreePct, pre vSphere 5.0, was the percentage of memory set aside by the VMkernel to ensure there are always sufficient system resources available. I received a question on twitter yesterday based on the explanation in the vSphere 5.1 Clustering Deepdive and after exchanging > 10 tweets I figured it made sense to just write an article.

@FrankDenneman @DuncanYB reviewing vSphere 5.1 Clustering Deepdive and have difficulties with explanation of Memory shares on pages 160-161

— Tim Jabaut (@vmcutlip) June 13, 2013

Mem.MinFreePct used to be 6% with vSphere 4.1 and lower. Now you can imagine that when you had a host with 10GB you wouldn’t worry about 600MB being kept free, but that is slightly different for a host with 100GB as it would result in 6GB being kept free but still not an extreme amount right. What would happen when you have a host with 512GB of memory… Yes, that would result in 30GB of memory being kept free. I am guessing you can see the point now. So what changed with vSphere 5.0?

In vSphere 5.0 a “sliding scale” principle was introduced instead of Mem.MinFreePct. Let me call it “Mem.MinFree”, as I wouldn’t view this as a percentage but rather do the math and view it as a number instead. Lets borrow Frank’s table for this sliding scale concept:

Percentage kept free of –>
Memory Range
6% 0-4GB
4% 4-12GB
2% 12-28GB
1% Remaining memory

What does this mean if you have 100GB of memory in your host? It means that from the first 4GB of memory we will set aside 6% which equates to ~ 245MB. For the next 8GB (4-12GB range) we set aside another 4% which equates to ~327MB. For the next 16GB (12-28GB range) we set aside 2% which equates to ~ 327MB. Now from the remaining 72GB (100GB host – 28GB) we set aside 1% which equates to ~ 720MB. In total the value of Mem.MinFree is ~ 1619MB. This number, 1619MB, is being kept free for the system.

Now, what happens when the host has less than 1619MB of free memory? That is when the various memory reclamation techniques come in to play. We all know the famous “high, soft, hard, and low” memory states, these used to be explained as: 6% (High), 4% (Soft), 2% (Hard), 1% (Low). FORGET THAT! Yes, I mean that… forget these as that is what we used in the “old world” (pre 5.0). With vSphere 5.0 and up these water marks should be viewed as a Percentage of Mem.MinFree. I used the example from above to clarify it a bit what it results in.

Free memory state Threshold in Percentage
Threshold in MB
High water mark Higher than or equal to Mem.MinFree 1619MB
Soft water mark 64% of Mem.MinFree 1036MB
Hard water mark 32% of Mem.MinFree 518MB
Low water mark 16% of Mem.MinFree 259MB

I hope this clarifies a bit how vSphere 5.0 (and up) ensures there is sufficient memory available for the VMkernel to handle system tasks…

Share it:

  • Tweet

Related

Server 5.0, 5.1, memory, resource management, VMware, vSphere

Reader Interactions

Comments

  1. Anthony Elizondo says

    14 June, 2013 at 16:32

    A graph of the piecewise linear function to calculate Mem.MinFree http://www.wolframalpha.com/input/?i=Plot%5BPiecewise%5B%7B%7Bx*.06%2C0%3C%3Dx%3C4096%7D%2C+%7B245.76%2B%7Bx-4096%7D.04%2C4097%3C%3Dx%3C%3D12288%7D%2C+%7B573.44%2B%7Bx-12288%7D.02%2C12289%3C%3Dx%3C%3D28672%7D%2C%7B901.12%2B%7B%7Bx-28672%7D*0.01%7D%2C28673%3C%3Dx%7D%7D%5D%2C+%7Bx%2C+0%2C+102400%7D%5D Just increase the range at the end for boxes with more than 100GB of RAM.

  2. Joris Van der Schoot says

    20 June, 2013 at 12:16

    Hi, in all the examples I’ve found, the 6 % value is used for the Mem.MemMinFreePct-setting. But what if you change this value? With the 6 %, the other percentages of the ranges are 4 %, 2 % and 1 %… But what if you change the 6 % to 10 % (for example!) ? Will the other percentages change too? I assume that the 4, 2 and 1 % are an outcome of 64 % of 6 % (which is 4 %), 32 % of 6 % (which is 2 %) and 16 % of 6 % (which is 1 %)…

    So changing the Mem.MemMinFreePct-setting to 10 % would, following the above assumption, result in:
    Range %
    4-12 GB => 6 % (64 % of 10 %)
    12-28 GB => 3 % (32 % of 10 %)
    remaining => 1.6 ~ 2 % (16 % of 10 %)

    Is this assumption correct? Or how does it work?

    • Duncan Epping says

      20 June, 2013 at 17:38

      you are correct in a way, mixing two things here. those ranges are only used with 5.0 which doesn’t use the “fixed” 6% any longer, it uses 6/4/2/1 all the way.

      With 4.1 and prior, changing that 6% would indeed lead to (4/2/1 aka 64/32/16) changing as well.

      • Joris Van der Schoot says

        21 June, 2013 at 10:43

        Thanks for your quick response. Ok, so changing the 6 % in 5.0 and after, changes the rest of the sliding scale too.

        Does the counter ‘Mem.LowFreeThreshold’ has a relation to the MemMinFreePct? When I use the 4.1-method for calculating the high threshold, the result equals the value mentioned in the LowFreeTreshold counter in stead of the total high treshold with the >5.0-sliding scale calculation …

        • Joris Van der Schoot says

          21 June, 2013 at 10:59

          I’m using 5.1

          • KiOng says

            25 July, 2013 at 16:29

            I think Duncan has answered that. Meaning in 4.1, if you change it will change the rest of the %. In vSphere 5.0 and above, it is fixed 6/4/2/1 all the way. If I understand correctly. It doesn’t change.

  3. Joel Hurford says

    26 June, 2013 at 22:34

    Would you clarify when each state is invoked? I have four memory states, which means I should only need three thresholds.

    High : greater than minfreepct
    Soft : between 64% minfreepct and minfreepct
    Hard : between 32% minfreepct and 64% minfreepct
    Low : less than 32% minfreepct

    Does anything different happen for memory reclamation around 16% minfree pct versus below 32%? What’s different in terms of memory reclamation techniques at 20% minfreepct versus 10% minfreepct? The table in the article doesn’t clarify the the “between this and that” for memory state.

    Thanks,

    Joel

    • Duncan Epping says

      27 June, 2013 at 18:19

      Frank explained that in his article here:
      http://blogs.vmware.com/vsphere/2012/05/memminfreepct-sliding-scale-function.html

      • Joel Hurford says

        27 June, 2013 at 19:44

        Thanks Duncan (btw I’m a fan of “HA and Clustering Deepdive”). I left a request at Frank’s blog too. The math still doesn’t work (as far as I can tell):

        High is greater than MinFreePct
        Soft is between 64% MinFreePct and MinFreePct
        Hard is between 32% MinFreePct and 64% MinFreePct
        Low is below 32% MinFreePct
        What memory state is below 16% MinFreePct? I think I am out of choices.

        • Duncan Epping says

          27 June, 2013 at 22:50

          Not sure I am following it… but this is what happens and when:
          Between High and Soft (64%): Ballooning
          When “Hard” water mark(32%) is reached: Ballooning, Compression, Swapping
          When Low water mark (16%) is reached: Ballooning, Compression, Swapping + “blocking of execution of VMs that consume more than their target size.”

          • Joel Hurford says

            27 June, 2013 at 23:39

            Thanks for your patience. I think we are almost there. I think my host is in a HIGH memory state when above MinFreePct. When I drop below MinFreePct, then I go to SOFT. When I drop below 64% MinFreePct, I should be in HARD. When I drop below 32% MinFreePct, I should be in LOW. You have another threshold at 16%, but I think I am out of memory states. You said ‘when HARD water mark (32%) is reached’. Didn’t I reach HARD when I fell below 64%? So when I get to the next threshold (32%), aren’t I due to be in LOW?

            I don’t see how I can start in HIGH, pass three thresholds and not be in the fourth and final state.

            Again, I appreciate you sticking with me. One of us is having a counting problem (maybe I need more fingers).

          • Joel Hurford says

            28 June, 2013 at 01:17

            Maybe my mistake is in understanding the transition from HIGH to SOFT. Am I in HIGH through the MinFreePct all the way to 64% of MinFreePct? I’ve been assuming the transition takes place at MinFreePct.

          • Duncan Epping says

            28 June, 2013 at 08:59

            Correct

  4. Stephane says

    20 February, 2014 at 12:09

    Hi Duncan,

    do the same principles (Mem.MinFreePct & Thresholds) apply to resource pools ?

Primary Sidebar

About the author

Duncan Epping is a Chief Technologist in the Office of CTO of the HCI BU at VMware. He is a VCDX (# 007) and the author of multiple books including "vSAN Deep Dive" and the “vSphere Clustering Technical Deep Dive” series.

Upcoming Events

04-Feb-21 | Czech VMUG – Roadshow
25-Feb-21 | Swiss VMUG – Roadshow
04-Mar-21 | Polish VMUG – Roadshow
09-Mar-21 | Austrian VMUG – Roadshow
18-Mar-21 | St Louis Usercon Keynote

Recommended reads

Sponsors

Want to support us? Buy an advert!

Advertisements

Copyright Yellow-Bricks.com © 2021 · Log in