cloud

Error during creation of NAT Routed network via VMware vCloud Director (vCD)

Duncan Epping · Sep 10, 2010 ·

Internally I have seen this one a couple of times so I knew what the issue was, but outside of VMware not many people have played with VMware vCloud Director (vCD) yet. Here’s the full error that is shown when you create a NAT Router Org network of vApp network:

Error creating Shield network appliance.
– vClould-Shield edge error: Creating/configuring the VR failed: vsmHandle.initializeEdge() net:1948253845/dvportgroup-218 vse:vm-220 VSM IP:10.0.0.10 failed.
– HTTP/1.1 403 Forbidden – The user does not have permission to perform this operation.

This usually means that the vShield Edge license key has not been added to vCenter. You can simply add it as follows:

From a vSphere Client host that is connected to a vCenter Server system, select Home > Licensing.
For the report view, select Asset.
Right‐click a vShield asset and select Change license key.
Select Assign a new license key and click Enter Key.
Enter the license key, enter an optional label for the key, and click OK.
Click OK.
Repeat these steps for each vShield component for which you have a license.

That should resolve this issue. Yes I agree, the error could have been more “user friendly” and I will ask the Engineering team if they can change this.

vCD – Networking part 2 – Network Pools

Duncan Epping · Sep 9, 2010 ·

In Part 1 of this series we described the different layers of networking. We already briefly spoke about Network Pools and that both a vApp Networks and Org Networks consume a segment out of a pool. In order to fully understand vCD (VMware vCloud Director) networking we will need to explain the foundation for all Cloud Internal traffic first which is the Network Pool.

Network Pools

As described in one of my earlier articles vCD mainly revolves around pooling of resources. This also goes for networking and more specifically for the Org Network and the vApp Network. Each vApp and Org Network will consume a network (segment) out of a defined pool. This network pool typically isolates network “segments” on layer 2 from the other networks in the pool. There are currently 3 types of network pools:

VLAN Backed
vCloud Network Isolation Backed (VCNI)
Portgroup Backed

Each of these pools have specific requirements, recommendations and constraints and they will be listed below. However all of the three different types of pools are interchangeable; with that meaning a segment from a VLAN-Backed Pool and a Portgroup-Backed pool each offer the same functionality to your Org and vApp network and basically should be seen as a means of intra-Cloud transportation. Or to simplify it even further, it provides a wire. This wire enables VMs in a vApp to communicate to each other or vApps in an Org to communicate to each other. This also means, and that might sound very logical to some, the distributed vSwitch / Nexus / vSwitch needs physical uplinks in order to enable communication across multiple hosts in a cluster!

When is a segment of the network pool used? The answer is simple, whenever you create a vApp or Org Network which is not directly connected to the above layer a segment of your designated network pool will be used. Before you ask, the reason the network pool is not used in the case of a “directly” connected Org or vApp network is because the Org/vApp network is just a logical object in that case and the VM/vApp ends up directly in the portgroup the layer above. I will show you what that looks like in Part 3 of this series which will tie part 1 and 2 together including vShield Edge.

Now lets assume you created a VLAN-backed or a vCloud Network Isolation-backed pool. When an isolated or NAT routed vApp/Org network is created vCD will automatically instantiate a new network segment. This segment is essentially a portgroup on a distributed vSwitch. This portgroup will provide layer two isolation between the different segments. As a distributed vSwitch is used every host in your cluster will instantly have the same portgroups available. There is a very specific reason I only mentioned the VLAN Backed and the VCNI Backed pool, for the Portgroup Backed pool you will need to manually pre-create all portgroups and of course ensure all those portgroups are available on every host in the cluster.

To visualize it a bit more I took three screenshots. The first one shows the network pool. The pool is VLAN backed and holds VLAN 1000 through 1100 and these networks will be created on the distributed vSwitched labeled as “dvSwitch”:

Now that we have created a network pool we can create an isolated Org Network, in this case we create a fully isolated Org Network. This isolated org Network aka “Internal network” enables the vApps within your virtual datacenter to communicate to each other:

In order to be able to communicate we need a network segment. This network segment is provisioned from the Network Pool we created earlier, which was VLAN Backed.

After you fill out some of the other details and click “finish” a portgroup is automatically created within vSphere. Although difficult to read the portgroup name contains the Org Network and the VLAN ID. (VLAN 1008 and Org Network “YB-ISO-ORG”). You can even see the task at the bottom of the screenshot:

Hopefully that visualizes the process a bit more. I dropped some steps for the creation of the pool and the Org Network to keep it relatively simple. You can find a full detailed video here. Now as stated earlier each of the the three types of pools have very specific requirements and constraints I listed them below so you can make a decision based on the requirements or constraints you might have.

VLAN Backed

VLAN-backed network pools require availability of a set of unused VLANs. When an Org or vApp network is created which is based on a VLAN-backed network pool a portgroup is created on a dvSwitch and a VLAN is assigned to this portgroup. It should be noted that all VLANs specified for the pool will need to be trunked to the host and that in most environment the amount of available VLANs is limited.

Requirements: Distributed vSwitch, pool of available VLANs, Physical uplinks need to support VLAN Trunks.
Recommendations: n/a
Constraints: Nexus 1000v and regular vSwitches are not supported currently, amount of available VLANs in most environments.

vCloud Network Isolation Backed

vCloud Network Isolation-backed(VCNI) network pools are flexible, easy to configure and do not require VLANs. vCNI provides layer 2 isolation by utilizing a network overlay. This network overlay is provided by MAC in MAC encapsulation and requires a vDS. For each consumed network vCloud Director creates a portgroup and assigns this portgroup a network ID number. This network ID number is used for the encapsulation of the traffic. As explained vCD uses MAC in MAC for the encapsulation of traffic. However, because of that the use of VCNI requires an increase in the MTU of the underlying transport network(dvSwitch) to avoid frame fragmentation due to the minor overhead cause by the MAC in MAC encapsulation.

Now this is a very thin explanation, but I don’t want to go to deep into this as it would only complicate things.

Requirements: Distributed vSwitch
Recommendations: minimum of 1 VLAN, MTU Increase (24Bytes, 1500 –> 1524).
Constraints: Nexus 1000v and regular vSwitches are not supported currently.

Portgroup Backed

Portgroup-backed pools require pre-created portgroups within the vSphere environment. Portgroup-backed pools are in my opinion the least flexible of all three options. However, Portgroup-backed pools do not require vDS and can be based on vSS, vDS or Cisco Nexus 1000v which can be a specific customer or even platform requirement.

Requirements: All portgroups need to be pre-created and available on all hosts of your cluster
Recommendations: Use a scripted solution or host profiles to create the portgroups to ensure consistency
Constraints: n/a

Wrapping up

Hopefully this clarifies network pools a bit. As explained, when creating and Org of vApp network which is not directly connected to the layer above a network of your Network Pool will be used to provide intra-cloud communication. For those who haven’t used vCD at all, believe me it took me a while to grasp these concepts so don’t be shy and ask questions/comment if anything is unclear.

vCD – Networking part 1 – Intro

Duncan Epping · Sep 7, 2010 ·

After my introduction on vCD last week, I thought it was time to publish an article on Networking. Networking is most likely the most complex concept of vCD(VMware vCloud Director) and can at times be very confusing. I have created three articles which will explain the concepts of networking within vCD and of course will explain on a technical level how things work. (Including the vSphere layer)

If there are any questions don’t hesitate to leave a comment. Please note that I am deliberately trying to simplify things in this first article as I don’t want you to get lost in any of the layers of networking vCD offers.

Layered

Networking within vCD is built up out of 3 distinct layers.

External Network
Org Network
vApp Network

These three layers have been created to give the end-user the flexibility needed in a multi purpose virtual datacenter. I have depicted all three layers in the following diagram which shows the logical relationship between the layers:

Some of you technical guys might say, that’s nice but I would like to see something less abstract. I created the following diagram which depicts the different layers in a different way. The diagram shows the three layers. I created a single External Network which links to two Org Networks. These Org Networks in its turn connect to multiple VMs(Org Y) and multiple vApps(Org X).

This is just an example however that illustrates possible network connections and as can clearly be seen it can be rather complex. As demonstrated there are multiple ways to connect vApps to each other or the outside world.

Now that we know some of the basics I will break down the three layers of networking. The first before we will discuss any of the advanced options like vShield Edge or network pools

External Network

The External Network is used for inter-Cloud connections. Or as I like to call it “your connection to the outside world”. It is the first network “object” that you create within vCD. An External Network is always backed by a Portgroup, meaning that a portgroup needs to exist within vSphere before you can create this vCD network object. This portgroup can be on a regular vSwitch, a dvSwitch or you could use Nexus 1KV. It all works, and all of them are supported!

Of course it is heavily recommended to set this portgroup up with a VLAN for layer 2 isolation, again note that this is an outbound facing connection for your Org or for multiple Orgs.

Examples of External Networks are:

VPN to customer site
Internet connection

As said, an external network can be shared between organizations but is typically created per organization and is your connection from or to your virtual datacenter.

I would to stress that, the external network is your exit from your virtual datacenter or your entrance!

Org Network

The second object that is created is the Org Network. The Org Network is used for intra-Cloud connections. Or as I like to call it “Cloud internal traffic”. An Org Network is linked to an organization and can be:

Directly connected to an External Network
NAT/Routed connected to an External Network
Completely Isolated

With that meaning that although an Org Network is primarily intended for internal traffic, it can be linked to an External Network to create an entry to or exit from your virtual datacenter. Note that it doesn’t necessarily need to be connected to an External Network, it could be completed isolated and used for “Cloud internal traffic” only! A use case for this would be for instance a test/dev environment where vApps will need to communicate with each other but not with the tenants back-end.

It should also be noted that the Org Network is mandatory! Every organization needs an Org Network, it is the only mandatory network object.

Just for completeness, an Org Network consumes a segment from a Network Pool when it is NAT/Routed or Isolated. A network pool is a collection of L2 networks which will be automatically consumed by vCD when needed, and what I call a segment is one of those L2 networks… basically a portgroup. I will explain Network Pools more in-depth in part 2.

When an Org Network is directly connected it is just a logical entity and physically doesn’t exist. Again in one of the following articles(part 3) I will explain what that looks like in vCenter.

vApp Network

The vApp Network kind of resembles the Org Network as it also consumes a segment from a Network Pool. The vApp Network enables you to have a vApp internal network, this could be useful for isolating specific VMs of a vApp. The vApp Network can be:

Directly connected to an Org Network
NAT/Routed to an Org Network
Completely Isolated

It should be noted that the “directly connected” option for both the Org Network and the vApp Network is just a logical connection. In the back-end it will be directly connected to the layer above.

As shown in an earlier diagram and explained above a vApp can contain multiple networks. This can be used to isolate specific VMs from the outside world. An example is shown in the following diagram where only the Web Server has a connection to the Org Network and the App and Database servers are isolated but do connect to the Web server.

Summary

vCD has three different layers of networking. Each of these layers has a specific purpose. The External Network is your connection to the outside world, the Org Network is linked to a specific Organization and the vApp network only resides within a vApp.

That is it for Part 1. Part 2 will focus on the Network Pools and Part 3 will focus on what these vApp, Org and External Networks look like on a vSphere layer and some general best practices.

My tip of the day, if you want to get to know vCD really well, check vCenter every time you make a change and see what happens!

UPDATE: for a full schematic overview check Hany’s awesome diagram.

VMware vCloud Director (vCD)

Duncan Epping · Aug 31, 2010 ·

As many of you know months ago I moved from VMware Professional Services to the VMware Cloud Practice. A major part of our work revolves around VMware vCloud Director(vCD) so you can imagine that I am glad it has finally been released. This lifts the NDA and as such you can expect a whole bunch of articles in the near future about vCD.

What is?

VMware vCloud Director is a new abstraction layer. vCD, as I will refer to it as of now, is a layer on top of vCenter and abstracts all the resources vCenter manages. All these resources are combined into large pools for your customers consume or should I call them tenants which seem to be the cool term these days. VMware vCloud Director does not only abstracts and pools resources it also adds a self service portal. As stated before it is more or less bolted on top of vCenter/ESX(i). I created a diagram to visualize this a bit more. Please note that this is still a simplistic and high level overview:

Now, I guess you noticed it says “VMware vCloud Director Cluster”. This cluster is formed by multiple vCD servers or as we refer them to “cells”. The cells form vCD and are responsible for the abstraction of the resources and the portal amongst other features that will be discussed later.

As stated before, vCD abstracts resources which are managed by vCenter. There are currently three types of resources that can be used by a tenant. Below each of the resource types I have mentioned what it links to on a vSphere layer so that it makes a bit more sense:

Compute
– clusters and resource pools
Network
– dvSwitches and/or portgroups
Storage
– VMFS datastores and NFS shares

These resources will be offered through a self-service portal which is part of vCD. As a vCD Administrator you can use the vCD portal to carve up these resources as required and assign these to a customer or department, often referred to in vCD as an “Organization”. Please note here that vCD is not purely designed for Service Providers, vCD is also designed for Enterprise environments.

In order to carve up these resources a container will need to be created and this is what we call a Virtual Datacenter. There are two different types of Virtual Datacenter’s:

Provider Virtual Datacenter (Provider vDC)
Organization Virtual Datacenter (Org vDC)

A Provider Virtual Datacenter is the foundation for your Compute Resources. When creating a Provider Virtual Datacenter you will need to select a resource pool, however this can also be the root resource pool aka your vSphere cluster. At the same time you will need to associate a set of datastores with the Provider vDC, generally speaking this will be all LUNs masked to your cluster. Some of my colleagues described the Provider vDC as the object where you specify the SLA and I guess that explains the concept a bit more. So for instance you could have a Gold Provider vDC with 15K FC disks and N+2 redundancy for HA while your Silver Provider vDC just offers N+1 redundancy and runs on SATA disk… everything is possible.

After you have created a Provider vDC you can create an Org vDC and tie that Org vDC to a vCD Organization. Please note that an Organization can have multiple Org vDCs associated to it. I depicted this in the following diagram, there are three Org vDCs owned by a single Organization across two Provider vDCs. The two provider vDCs each have a specific SLA.

So what can I do with these Org vDCs? Simply said: consume them. You can create vApps, and a vApp is just a logical container for 1 or more virtual machines. This vApp could for instance contain a three tiered app which has an internal network and a firewalled outbound connection for a single VM, which would look something like this:

Of course there are various ways to create a network, but that is way to complex for an introduction article. I will go into more details around all the cool networking functionality that is offered in one of the following articles however.

As you can see there is a lot possible with vCD, I guess too much to describe in a single article.

To summarize, vCD offers a self service portal. This portal enables you to provision resources to a tenant and enables the tenant to consume these resources by creating vApps. vApps are a container for one or multiple virtual machines and can contain isolated networks. As said, there is a lot more to vCD but I feel that all of you should play around with it a bit more before I dive into some of the specifics. (For those at VMworld, LAB 13!)

As you can imagine I am really excited about this release, and am absolutely thrilled that I can finally talk about this. There are more blog articles coming up, but I just want the dust to settle a bit first so everyone can see those clouds!

More background/download links can be found here:

Release Notes:
http://www.vmware.com/support/vcd/doc/rel_notes_vcloud_director_10.html

Download Landing Page:
http://downloads.vmware.com/d/info/datacenter_downloads/vmware_vcloud_director/1_0

Documentation Landing:
http://www.vmware.com/support/pubs/vcd_pubs.html

Product site:
http://www.vmware.com/products/vcloud-director/

Eval Guide:
http://www.vmware.com/files/pdf/techpaper/VMW-vCloud-Director-EvalGuide.pdf

Screenshots:
http://www.vmware.com/products/vcloud-director/screens.html