Comments on: VMFS/LUN size? http://www.yellow-bricks.com/2009/06/23/vmfslun-size/ Building blocks for virtualization... Thu, 18 Mar 2010 19:25:38 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Erik http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-6775 Erik Tue, 23 Feb 2010 12:44:23 +0000 http://www.yellow-bricks.com/?p=3359#comment-6775 Ah! Of course, didnt think about SCSI reservations. I'll contact my array vendor for queue depth settings. Thanks for the help Duncan! Ah! Of course, didnt think about SCSI reservations. I’ll contact my array vendor for queue depth settings. Thanks for the help Duncan!

]]> By: Duncan http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-6772 Duncan Tue, 23 Feb 2010 07:06:34 +0000 http://www.yellow-bricks.com/?p=3359#comment-6772 Depends on the queue depth of the array, but in general you will have a per LUN queue depth which in this case will make a difference. Also think about SCSI Reservations. The more VMs on a single volume the bigger the chance is of SCSI Reservation conflicts. So there is a good reason to narrow the LUN size down. Depends on the queue depth of the array, but in general you will have a per LUN queue depth which in this case will make a difference. Also think about SCSI Reservations. The more VMs on a single volume the bigger the chance is of SCSI Reservation conflicts. So there is a good reason to narrow the LUN size down.

]]> By: Erik http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-6770 Erik Mon, 22 Feb 2010 23:10:21 +0000 http://www.yellow-bricks.com/?p=3359#comment-6770 First of all; sorry for commenting on a really old article - we all hate necromancy but I just have to ask: Everyone keeps talking about the need for several LUN's. What if I have a 2TB RAID10 array (4x2TB SATA)? Am I supposed to create 4x500GB LUNs (or whatever LUN size I choose to use)? Does this matter at all when every LUN is hitting the same spindles anyway? I guess it's not a normal scenario since many use SAS/FC disks, but I'm not in that budget range. First of all; sorry for commenting on a really old article – we all hate necromancy but I just have to ask:

Everyone keeps talking about the need for several LUN’s. What if I have a 2TB RAID10 array (4×2TB SATA)? Am I supposed to create 4×500GB LUNs (or whatever LUN size I choose to use)? Does this matter at all when every LUN is hitting the same spindles anyway?

I guess it’s not a normal scenario since many use SAS/FC disks, but I’m not in that budget range.

]]> By: nate http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3979 nate Wed, 08 Jul 2009 00:11:29 +0000 http://www.yellow-bricks.com/?p=3359#comment-3979 Whoops minor correction, 14.1TB configured, 1.82TB is used(Before RAID), 2.2TB is used(after RAID, QA+Corp RAID 5 5+1, Prod operations RAID 5 3+1) Whoops minor correction, 14.1TB configured, 1.82TB is used(Before RAID), 2.2TB is used(after RAID, QA+Corp RAID 5 5+1, Prod operations RAID 5 3+1)

]]> By: nate http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3978 nate Wed, 08 Jul 2009 00:08:26 +0000 http://www.yellow-bricks.com/?p=3359#comment-3978 For my systems I use 1TB lun sizes, they are striped across 200 SATA disks in my storage array(VMFS volumes make up less than 1% of my overall storage as far as disk space and IOPS). I have 12 VMFS data volumes today(3 for Prod OPS, 5 for QA, 4 for Corp/IT). I have 1 swap VMFS volume for each environment(primarily to control space provisioning and monitor swap IOPS). All in all 14.1TB configured, with thin provisioning 2.2TB is in use right now. Have about 110 VMs currently. Databases are using RDM primarily for array based snapshots/replication between systems. I have decided to mark my first two production VMFS volumes "full", VMware has provisioned about 1TB between the two. The extra space is room to grow for existing VMs. Since it is thin provisioned really no space is wasted, I don't care if the volume has 400GB of unused capacity because it really is unused capacity. IOPS wise our production operations VMFS volumes as a whole average 170 IOPS(Servers total 40 cores/283GB ram). For Corp/QA their volumes average 248 IOPS(Servers total 128 cores/512GB ram). I've grouped the volumes on the array into storage pools for QA and for corp, so I can monitor/alert on their space usage as a group rather than have to monitor the volume levels individually. I'm also in the process of migrating all systems to a boot from SAN configuration. I have 6 systems booting from SAN today(Fiber), the rest will be converted in the next month or two as we upgrade to vSphere. ESXi v4 is taking 1.5GB of space(so much for that 70MB number they toss around, I hate misleading things and that number is horribly misleading), ESX v4 is taking 3.5GB(space is allocated in 16kB increments on the fly as data is written). Everything is fiber connected. For my systems I use 1TB lun sizes, they are striped across 200 SATA disks in my storage array(VMFS volumes make up less than 1% of my overall storage as far as disk space and IOPS). I have 12 VMFS data volumes today(3 for Prod OPS, 5 for QA, 4 for Corp/IT). I have 1 swap VMFS volume for each environment(primarily to control space provisioning and monitor swap IOPS). All in all 14.1TB configured, with thin provisioning 2.2TB is in use right now. Have about 110 VMs currently. Databases are using RDM primarily for array based snapshots/replication between systems. I have decided to mark my first two production VMFS volumes “full”, VMware has provisioned about 1TB between the two. The extra space is room to grow for existing VMs. Since it is thin provisioned really no space is wasted, I don’t care if the volume has 400GB of unused capacity because it really is unused capacity.

IOPS wise our production operations VMFS volumes as a whole average 170 IOPS(Servers total 40 cores/283GB ram). For Corp/QA their volumes average 248 IOPS(Servers total 128 cores/512GB ram).

I’ve grouped the volumes on the array into storage pools for QA and for corp, so I can monitor/alert on their space usage as a group rather than have to monitor the volume levels individually.

I’m also in the process of migrating all systems to a boot from SAN configuration. I have 6 systems booting from SAN today(Fiber), the rest will be converted in the next month or two as we upgrade to vSphere. ESXi v4 is taking 1.5GB of space(so much for that 70MB number they toss around, I hate misleading things and that number is horribly misleading), ESX v4 is taking 3.5GB(space is allocated in 16kB increments on the fly as data is written).

Everything is fiber connected.

]]> By: rbrambley http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3942 rbrambley Thu, 02 Jul 2009 12:52:28 +0000 http://www.yellow-bricks.com/?p=3359#comment-3942 Duncan, Now with vSphere's ability to virtualize any application more VMs with larger amounts of RAM could impact VMFS design. That means either more RAM reservations or more disk space needed for larger VM swap files. Assuming most admins will either forget or decide not to make a reservation, couldn't your formula be adjusted to round(((maxVMs * avgSize) + 20%)+total vRAM )? I know the 20% has accounts for the vRAM to date, but I'd leave it in the formula "just in case." Duncan,

Now with vSphere’s ability to virtualize any application more VMs with larger amounts of RAM could impact VMFS design. That means either more RAM reservations or more disk space needed for larger VM swap files.

Assuming most admins will either forget or decide not to make a reservation, couldn’t your formula be adjusted to round(((maxVMs * avgSize) + 20%)+total vRAM )? I know the 20% has accounts for the vRAM to date, but I’d leave it in the formula “just in case.”

]]> By: ozwil http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3940 ozwil Thu, 02 Jul 2009 12:28:02 +0000 http://www.yellow-bricks.com/?p=3359#comment-3940 Duncan Is there any performance loss or gain by having a large VMFS datastore spanning multiple smaller LUNs Cheers Duncan

Is there any performance loss or gain by having a large VMFS datastore spanning multiple smaller LUNs

Cheers

]]> By: Duncan http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3937 Duncan Thu, 02 Jul 2009 10:02:10 +0000 http://www.yellow-bricks.com/?p=3359#comment-3937 I like to keep it together. 1) it's easier 2) mixed i/o profile 3) in case of SRM/DR, one location for a single VM I like to keep it together.
1) it’s easier
2) mixed i/o profile
3) in case of SRM/DR, one location for a single VM

]]> By: houghtp http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3936 houghtp Thu, 02 Jul 2009 08:23:34 +0000 http://www.yellow-bricks.com/?p=3359#comment-3936 How do people design their VMFS at a VM level i.e. do you put all OS in one datastore, page file in another and data in another or do you just keep VM's disks together. At the moment we split our VM's vmdk's out and i'm starting to think we should go back to keeping the disks all together for the majority of VM's, it would make admin/scripting/backups/replication so much easier. How do people design their VMFS at a VM level i.e. do you put all OS in one datastore, page file in another and data in another or do you just keep VM’s disks together.

At the moment we split our VM’s vmdk’s out and i’m starting to think we should go back to keeping the disks all together for the majority of VM’s, it would make admin/scripting/backups/replication so much easier.

]]> By: Duncan http://www.yellow-bricks.com/2009/06/23/vmfslun-size/comment-page-1/#comment-3891 Duncan Sun, 28 Jun 2009 07:19:16 +0000 http://www.yellow-bricks.com/?p=3359#comment-3891 It depends indeed. There are several reasons for including or excluding these, I would say Capacity Planner input is one of those :-) It depends indeed. There are several reasons for including or excluding these, I would say Capacity Planner input is one of those :-)

]]>