Part 1 explained the basic concepts of networking within vCD(VMware vCloud Director) and Part 2 focussed on Network Pools. In the final and 3rd part we will focus on a use case and what happens on the vSphere layer with these different types of vCD networks. I will cover just a single use case for now, but this one basically covers all areas! Please read both part 1 and part 2 of this series before you start reading this part. Lets just start diving into a scenario.
vApp directly connected to an External routed Org Network
Use cases:
- Internet connection for the VMs in your virtual datacenter. Firewall can be enabled to block all incoming traffic.
- Publicly publishing a single “service” externally by enabling NAT on the vShield Edge device. In this case all incoming traffic will be blocked and only a single IP will be translated and route back to that particular VM.
We will start with the basics. The flow of the network in this case will be:
As explained in Part 1 the External Network is backed by a Portgroup. This portgroup can be a regular portgroup on a vSwitch, or one on a dvSwitch or even the Nexus 1Kv. We will start by creating a dvPortgroup.
External Network
Lets first create a dvPortgroup within vCenter. This is the dvPortgroup that the External Network will use. We will give it a VLAN ID for layer 2 isolation. In this case we use VLAN ID 105 and label the dvPortgroup as “dvExternal-105”.
Now we will need to create a network within vCD that enables your vApp and Organization to use this dvPortgroup we just created. We do this by creating an “External Network”. (option 3 on your home screen in vCD.) First we will need to select the correct dvPortgroup we created:
Next thing to do is specify the associated IP Range, Gateway, Netmask etc. The IP-Range is used for any VMs that are directly connected to this External Network and for the vShield Edge devices. But we will show that later in this article.
Next up is is giving the External Network a name, we will keep it simple and name it “External – vlan 105”:
That is it for the External Network part. Now lets create an Org Network.
Org Network
We will create an External Org Network which is routed to an External Network. (On your home screen go to “7 Add another network to an organization”.) Select the Organization it needs to connect to first and then the real magic starts.
We will use the typical setup. We have unticked “Create an internal network”, and we have selected “Routed connection”:
The cool thing about the network section of vCD is that is shows you what it is building. In this case you can see that the vApp is directly connected to the External Org Network (NAT-Routed) which in its turn is connected to the External Network through a vShield Edge device. The next step is to select the correct External Network that this External Org Network connects to:
Please note that we also have selected a network pool, in this case the vCloud Network Isolated Pool! Next we will need to specify the associated IP Range, Gateway, Netmask for the Org Network. Now you might think that we have already done this but that was for the External Network! The pool of addresses will be used for any device that sits within the Org Network boundaries.
Of course the final step is giving this Org Network a name:
vApp layer
As this post is about networking I will skip the creation of the vApp itself but will show you what we have done in a single screenshot. As this screenshot below shows the VM is directly connected to the Org Network labeled “YB-NAT-Ext-Org”:
Connecting the dots
Now that we have shown you how this is created within vCD you would probably want to know what this results in on a vSphere layer. When we created the Org Network a dvPortgroup was automatically created. This automatic creating was enabled by the use of a network pool. The network pool in this case was a vCloud Network Isolation backed network pool.
The screenshot below shows the dvPortgroup that represents the Org Network. The VM that was created called “Direct”, however vCD uses IDs to uniquely identify VMs and as such it is labeled as “1227504509-Direct” within vSphere. Please note the “F46” in the name of the dvPortgroup. This means that it is using a fenced network with ID 46. (fenced –> vCloud Network Isolation) This Network Pool happens to use VLAN 107 (V107), which was defined when the pool was created and is also shown in the screenshot below.
In order for VM “1227504509-Direct” to communicate to the outside world it will need a connection to the External Network. As shown and described above VMware vCloud Director uses vShield Edge to do this. In other words, the vShield Edge device will have multiple NICs. This is shown in the following screenshot. The External Network portgroup contains a vShield Edge device (vse-651240915) which is the same device as shown in the screenshot above.
This is the vShield Edge device that enables the VM “1227504509-Direct” to communicate with the outside world, as it is connected to both portgroups.
Traffic Flow
As it took me a while to understand how this worked, I have created a couple of diagrams. The first diagram shows all components we created and how they are linked:
I guess this is still not saying much. Lets add the flow of the traffic to this diagram by extending it with another vApp. What if you would have two vApps connected to the same Org Network and both VMs of these vApps are on a different host in your cluster and the first VM wants to connect to the second VM? What does the flow of traffic look like? As you can see in the diagram below the VM of the first vApp is connected to the same dvPortgroup. However as both vApps reside on a different host the traffic will need to go to the physical switch layer first:
The other scenario I wanted to show is where a vApp wants to connect to a device on the outside world. In this case I labeled it as “internet” but it could be anything. Also I have assumed that the vShield Edge device resides on a different host than the VM that wants to connect to the internet.
It took me a while to write this “use case”. I hope this makes vCD networking slightly better to understand… but again the key here is to play around with it. If there are any questions please don’t hesitate to reach out to me! If I can find the time I will write another “use case” or maybe I will ask some of the other guys in my team to do something similar.
Excellent write up Duncan! Thanks! I’m still a little fuzzy on where and why you would use a vCNI network pool over a VLAN-backed network pool. Trying to sort out the use cases in my head.
I’m also a little fuzzy on how the vCNI pool uses mac-in-mac encapsulation. If I understand it right it is to mask the IP addresses from one vApp to another, but why not use something like NAT’ing for this?
1) VCNI is primarily used when you have a limited amount of VLANs to spare for the Org and vApp network. So large enterprise customers and Service Providers will more than likely want to use it.
2) we are talking about layer 2 here, vShield Edge is not for L2 but for L4 firewalling or NAT’ing services.
Ok, I’m getting a better picture of it now. I also found the vCloud Director Security Hardening Guide and thumbed through that a bit. Has some good detail in there and looks like a good read. It helped clarify a few things for me.
Thanks again!
Duncan, I see all your blogs on vcloud are detailed and informative. I have a question for you and need help.
I was trying to setup a vcloud and iam stuck at a point where i am not able to allocate Network pool to Organization vDC’s.
In the create organization network wizard (screenshot 6 in your blog)
1. I can see external network and network pool lists only if i uncheck “only use networks that are accessible by this organization”.
2. If i proceed further and uncheck the box, i will see external networks and network pool list, but i cannot select both external network and corresponding network pool at the same time, as when i select the external network , network pool list disappears and if i select the network pool first, external network list disappears.
Can you point me where i should be looking to resolve this. I was struggling to get it work for last 2 days.
Also do we need to change anything on the physical switches where the esx hosts are connected in terms of Vlan ID’s that are getting created from the cloud .
Great posting! I totally understand that VCNI is going to be a great fit for larger setups. However, if I want to achieve QoS and ACL’s (that can be done through VLAN backed) in VCNI, how do we do it?
Also, looking at the current support of Nexus1KV on network pools, would you agree using it on the External network alone (for security, QoS and monitoring features) and use dVS backed vApp and Org Network with VCNI?
Thanks in advance
-Janakan
Good questions Janakan. I guess the question I would have in return is how you are going to do QoS or ACLs when you don’t know who or what will be using which VLAN in a Pool? It is rather dynamic so your QoS and ACLs will be end up being really generic.
I agree that the Nexus1KV for External Networks would make sense when additional QoS / Monitoring is requirement.
Thank you for the response. If I can assign a unique Network pool for each Org DC backed by VCNI with Optional VLAN ID, I should be able to do the ACL and QoS on the physical layer and then use Nexus1KV on the external network as we agreed upon. Is that feasible?
Thank you!
-Janakan
Yes that would work but would completely defeat the whole purpose of a pool wouldn’t it?
Agreed but I believe it will ensure each org (client in a service provider environment) is ensured that their network (both external and Org) is isolated at both physical and virtual layer. As a service provider, has to provision/allocate resource pool for each org, it may give them more control and manageability. Thoughts ?
-Janakan
Just wanted to let you know I really like this post. Well written use case, so please find the time to write another use case. 🙂
When it comes to cloud , you have several networks that get created on the fly and most of them are with in the cloud but the ones that are connected to external networks is my concern. what infrastructure changes do you fore see in the existing infrastructure at a datacenter, when you introduce cloud to the datacenter.
Use Case : Automatice provisioning of Webservers and then making them accessible over internet. This becomes complex when you have several organizations and has similar requirements. What all would you think need to be provisioned outside the cloud environment ahead. I guess lot of Public IP’s but this might be diffcicult . Are there solutions around this.
Can vedge be deployed as a router ? or must he do NAT all the time ?
Is there plans to make duplicate ip scheme capability into vlcoud, so to make it real multi-tenant capable?
Duncan, about your second traffic flow diagram : what if 2
Duncan, about your second traffic flow diagram : what if 2 VMs connected to 2 different port-groups on the VDS (2 different network pools on VCD),VSE VM is needs to be spooned on the fly right ? but on which ESX server is is spooned ? on VM1’s ESX or on VM2’s ESX ? what would be the traffic flow then and how this is predicted ?
what if routing is needed ?, i mean real routing not NAT , how will you automate the creation of external routers ? i mean not using VSE as a router ? on the sme token what if your DC needs load-balancing that do more then HTTP ? what if you need a FW that do more then L3/L4 filtering ? etc …
Than you would need some sort of external orchestration tool that has a hookin with your external routing device. You could use an orchestration engine that does this for you based on information provided manually, but no automated mechanism for that currently unfortunately
Is it possible to get dynamic ip, if vshield edge and vm client are different esx server. For example, I have 2 esx servers a, b. My VSE1 is sitting on server a but my vmclient is on server b. I choose VSE1 interface for the vmclient. When I turn on the dhcp service on my VSE1, is it possible to get dynamic ip from my vmclient?
Hi Duncan,
First of all, let me apologize for my actions.. I don’t have any excuses for what I did and will not give you any either.. I am truely sorry for my actions and hope you can forgive me for the same. I will provide due credits and acknowledgement to your articles that are posted on my blog site.. You have wonderful site here and hope that you come up with more such knowledgeable articles in the future.
Once again, sorry.
Thanks.
Yohan
Hi,
Is it possible to use 2 different port groups with same VLAN ID for External Network and Network Pool, if I am using vDS or vSS?
Thanks,
Raj