file services

vSAN File Services considerations

Duncan Epping · Apr 15, 2020 ·

I was looking into vSAN File Services this week as I had some customers asking about requirements and constraints. I wanted to list some of the things to understand about vSAN File Service as it is important when you are designing and configuring it. First of all, it is good to have an understanding of the implementation, well at least somewhat as vSAN File Services is managed/upgraded/update as part of vSAN. It is not an entity you as an admin don’t manage the appliance you see deployed. I created a quick demo about vSAN File Services which you can find here.

If you look at the diagram (borrowed from docs.vmware.com) above you can see that vSAN File Service leverages Agent/Appliance VMs and within each Agent VM a container, or “protocol stack”, is running. The protocol stack is what exposes the file system as an NFS file share. [Read more…] about vSAN File Services considerations

Scaling out your vSAN File Services Cluster

Duncan Epping · Apr 10, 2020 ·

This week I have been testing with vSAN File Services and one of the procedures I wanted to run through was scaling out my vSAN File Services cluster. In my case, I have a cluster of 5 hosts and what I want to do is add a host to my vSAN cluster, expand the vSAN Datastore and also grow my vSAN File Services cluster.

First of all, when you add a host into the cluster you need to make sure it is in maintenance mode. If if is not in maintenance mode then vSAN FS will instantly try to clone a vSAN File Services agent VM (FS VM) on to it and that process will fail as there’s no disk group yet. So make sure to place the host into maintenance mode before adding it to the cluster.

After you added it to the cluster, you have to create the disk group first. Claim all the disks that need to be part of the disk group and create the disk group. When you have done that you can take the host out of maintenance mode. Now the FS VM will be cloned and powered on. However, one thing you will need to do is expand the IP Pool for the vSAN FS Protocol Stack container. You can do this as follows:

Go to your cluster
Click on vSAN / Services
Go to File Service and click Edit on the right
Go to the IP Pool page by clicking Next twice
Add that additional IP address and DNS Name
Click Next / Finish

Now a new Protocol Stack Container can be instantiated in that new FS VM and your vSAN File Services cluster has been scaled out properly. I created a simple demo showing you what the process looks like, make sure to check it out below!

Introducing vSAN File Services as part of vSAN 7.0

Duncan Epping · Mar 17, 2020 ·

There was a great article by Cormac talking about vSAN File Services published last week. I had some articles planned, but hadn’t started yet, so when I saw the article I figured I would do something different. No point in rehashing what Cormac has already shared right. I figured I would shoot another demo, and do a brief write-up so people know what vSAN File Services is all about. In vSAN/vSphere 7.0 there’s now a new feature, and this is vSAN File Service. vSAN File Services can simply be enabled on a cluster level and provides you NFS 3 and 4.1 capabilities. The great thing about the solution is that you can create file shares and associate policies with the file shares. In other words, you can have a file share with RAID-1, RAID-5, or maybe even striping or stretched across locations when that is supported.

When you enable File Service, vSAN will deploy a number of “agent VMs” and these VMs/appliances are fully managed by vSAN. These agent VMs run Photon OS and have the file service capabilities enabled through docker/container technology. After these File Service Agent VMs have been deployed, the docker container instances have been instantiated and configured, vSAN File Service will be up and running and available for use. Next, you could simply create a file share and start consuming it. But before I reveal everything, let’s just head over to the demo below. I hope you enjoy it!

HCI3041BU: Introducing Scalable File Storage on vSAN

Duncan Epping · Sep 6, 2018 ·

Another beta announcement last week for vSAN was around Native File Services. This was the topic of HCI3041BU, which was titled “Introducing Scalable File Storage on vSAN with Native File Services”. The full session can be found here, the summary is below for your convenience. The session was by Venkat Kolli (Product Manager) and engineers Rick Spillane and Wenguang Wang.

Venkat kicks of the session describing the different types of storage most of our customers have in their data center today, and also what kind of data lands on the different types of storage. Basically, it is divided into three main types: Block, File, and Object. Where I personally believe that “object” is at the point of becoming more common on-premises but for many is consumed as a cloud service. Looking at where the data growth is today, it is mainly in the “unstructured data” space.

Next Venkat discusses the management complexity of traditional file storage, not just management complexity but also scaling and forecasting. Which in most cases leads to increased cost. How can vSAN help with simplifying File Services and lowering cost by providing a framework which allows you to serve block, file and object. For now, we are discussing file-services however, but the vision is clear.

Rick is up next introducing File Services. vSAN File Services allows you to create file shares and provide file services to users/consumers through the same familiar interface you have available today in vSphere. On top of that, you get to leverage the power of policy-based management to provision file shares in a specific way. Which means that File Shares will work in stretched clusters, can be protected with vSAN Data Protection, can be striped/replicated etc. Most important piece of feedback during the design phase from customers was that they did not want a separate storage cluster to manage for file services, this needed to be an integral part of today’s offering.

The requirements and design principles for the vSAN Distributed File System were:

Elastic Scaling
- Scale IOPs up/down
Single namespace across the cluster
Centrally managed, configured and monitored
Transparent failover
POSIX File Interface
Use vSAN services like data path, consensus mechanisms, and checksumming

Rick next explains a new platform that will (potentially) be included in vSAN, this is called the Storage Services Platform. What this provides is stateless containerized frontend servers which sit on top of the vSAN Distributed File System. This will be available for both VMware and partners, so even partners should be able to provide storage services through this platform. Data will sit in the VDFS volumes and then will be exposed through these services. These services, of course, are fully distributed and self-managing.

The Storage Services Platform is implemented in the form of a storage services control plane. This control plane will for instance monitor all front-end servers and node and help in the case of failures, but also will help to ensure availability during maintenance and upgrade. Also, when it comes to scalability the control plane monitors the instances and allows to scale up and down when needed.

Okay, that sounds great, but how do file shares get formed? File shares will be an aggregate of one or multiple vSAN Objects. The great thing about this is that it allows for elasticity in size and performance, plus policies can be associated with these objects. You can now simply create file shares through the UI, or leverage the API. The vSAN team made sure that you can access it and define them the way you prefer. On top of that, this platform will also be available to Kubernetes as part of our Cloud Native Storage Control Plane.

Next Rick briefly discussed data protection for file shares, he mentioned that the team has worked with various 3rd party vendors to allow for full backup and recovery, including file-level restore. What Rick also revealed, surprisingly enough, is that in the initial release we will have:

NFS v4.1 support
AD-based Authentication
Kerberos
Containerized application support

And in the release after that support for the following is planned:

SMB
vSAN DP Integration
OpenLDAP support

Next Wenguang came up on stage, and he demoed vSAN File Services. He showed how simple it is to enable File Services in the UI. Literally, a couple of steps, provide the networking details and also authentication mechanism. The next step will be to download an OVF, this contains the frontend service we spoke about earlier, for now, this is an NFS server, but this could be other services in the future. After the File Services have been enabled and the OVF is deployed you can start creating file shares. Again this is very straightforward, part of the familiar vSAN UI / HTML-5 interface, which is what I like most, if you know vSAN and/or vSphere you will be able to use vSAN File Services as well. I hope potential other services will be implemented in a similar easy manner.

The Q&A was interesting as well, as some questions around the potential SMB implementation were answered (SAMBA on Linux vs Microsoft vs Dell/EMC stack?) and for instance what block size is used for the file system (4K, like vSAN).

All in all a very exciting solution, and a great overview of what you can expect in the future for vSAN. Note that this is part of the beta, so if you are interested sign up!