Windows 2012 Hyper-v 3.0 Network Virtualization

This Blog post was inspired by a set of blog post by BrianEh at his blog http://ITProctology.blogspot.com  in effort to better understand Windows 2012 Virtualization.

A new feature in Hyper-v 3.0 is network virtualization which allow hosting providers to host VMs from different customer that can have the same IP address schema without the need to change their IP address or facing any IP conflict errors. Hyper-v provide two method to achieve this :

  • IP packet encapsulation protocol Generic Routing Encapsulation (GRE), that encapsulate VM traffic into hyper-v traffic and use the host IP to deliver the encapsulated traffic to its destination hosting hyper-v.
  • IP Rewrite : is Kind of Static Nat configuration where each VM IP address get rewritten using physical  IP address before it is transferred on the physical network.

The example below use GRE ,The following description was take form Simple Hyper-V Network Virtualization Script with Gateway

“Hyper-V Network Virtualization (WNV) virtualizes the VM networks by:
1. Associating each VM IP address (Customer Address, CA) with a corresponding physical IP address used on the actual packets on the wire (PA)

2. (Provider Address,PA). Each VM packet is transformed (rewritten or encapsulated) from a CA  packet to PA packet before the packet is transmitted onto the physical  network.

3.On the destination host, the packet is transformed back with matching rules from PA packet to CA packet before it is delivered to the destination VM.

The WNV module operates on the packets based on the virtualization policy rules provisioned from management servers such as SCVMM (System Center Virtual Machine Manager). These policy rules define the mapping of CA:PA for each VM, the corresponding virtualization mechanism (rewrite/encap), and which customer virtual subnet these rules are for. Additionally, the virtualization rules also defines routing topology between customer virtual  subnets, and between WNV virtual subnets and non-WNV networks (cross premise, virtual-to-physical resources, etc.) “

its So lets see the below Scenario where we will host two VMs for two different customers on the same Host using the same IP address.I used BLACK font for host1 configuration , RED for host 2 configuration and Blue for PowerShell output

The steps to enable and configure network virtualization:

1. Assign VMs to a Virtual network id

2. Enable network Virtualization

3. Assign Provider IPs

4. Create Domain and Define Customer Routes

5. Build Lookup table (Routing table) 

Win2k12Virt

1. Assign VMs to a Virtual network id 

To Start lets  Setup CustomerA-VM1 and CustomerB-VM1 on Host1 connecting them to the external Switch , if you assign IP address  to these Two Machines you would end with IP Conflict error on one these machines. An IPconfig/all would show this :

image

now you can solve this by assigning each VM to different VLAN , but you can only use maximum of 4,096 also , you would need to configure your switch for VLAN Tagging. Hyper-v 3.0 remove this limitation using network Virtualization .

So lets First Solve this issue at the level of single host , using the Get-VMNetworkAdapter PowerShell command , we can look at the IPs assigned to these two machines , in the output below note that  CustomerB-VM1 has two IP address one of them is APIPA address due to Address conflict

Get-VMNetworkAdapter cu*

Name            IsManagementOs VMName        SwitchName         MacAddress   Status IPAddresses
—-            ————– ——        ———-         ———-   —— ———–
Network Adapter False          CustomerB-VM1 LAN Virtual Switch 00155DFA9905 {Ok}   {192.168.50.1, 169.254.97.103, f…
Network Adapter False          CustomerA-VM1 LAN Virtual Switch 00155DFA9904 {Ok}   {192.168.50.1, fe80::2c30:3b37:8…

So to solve this Problem  assign each machine to a different Virtual LAN using the  Set-VMNetworkAdapter  commandlet , we will assign Customer A the VLAN ID 445566 and Customer B the VLAN ID  7788990 , you can notice that we have passed the 4096 limitation of the IEEE 802.1Q limitation . On Host1 Open PowerShell command prompt :

Get-VMNetworkAdapter customerA-vm1 | Set-VMNetworkAdapter -VirtualSubnetId 445566

Get-VMNetworkAdapter customerb-vm1 | Set-VMNetworkAdapter -VirtualSubnetId 7788990

you can now go and disable and re-enable Network adapter at CustomerB-VM1 and see that your IP address conflict have been cleared . you can check this again by reissuing the command:

Get-VMNetworkAdapter cu*

Name            IsManagementOs VMName        SwitchName         MacAddress   Status IPAddresses
—-            ————– ——        ———-         ———-   —— ———–
Network Adapter False          CustomerB-VM1 LAN Virtual Switch 00155DFA9905 {Ok}   {192.168.50.1, fe80::857b:2c47:a…
Network Adapter False          CustomerA-VM1 LAN Virtual Switch 00155DFA9904 {Ok}   {192.168.50.1, fe80::2c30:3b37:8…

Even Without enable network Virtualization on the host we where able to solve this conflict and by pass the VLAN Limitation problem. the same command need to be applied to VMs on Host2.

Get-VMNetworkAdapter customerA-vm2 | Set-VMNetworkAdapter -VirtualSubnetId 445566

Get-VMNetworkAdapter customerb-vm2 | Set-VMNetworkAdapter -VirtualSubnetId 7788990

2. Enable network Virtualization

But  VMs on different hosts will not be able to communicate with each other until we enable the Hyper-v network Virtualization part , also notice that these assigned   VLANS will not show in the Hyper-v GUI , to see to which VLAN each machine is assigned use the command :

Get-VMNetworkAdapter * | Format-Table VMName, Name, MACAddress, VirtualSubnetID, IPAddresses -AutoSize

VMName                 Name                            MacAddress   VirtualSubnetId IPAddresses
——                         —-                               ———-   ————— ———–
SQL2012-161            Network Adapter 00155DFA9900               0     {192.168.250.161}
SQL2008-160          Network Adapter 00155DFA9903               0      {192.168.250.160}
SCVMM-162             Network Adapter 00155DFA9902               0      {192.168.250.162}
SCSMMGM-164      Network Adapter 00155DFA9908               0       {192.168.250.164}
SCSMDW-165          Network Adapter 00155DFA9907               0       {192.168.250.165}
SCOM-163               Network Adapter 00155DFA9906               0        {192.168.250.163}
CustomerB-VM1     Network Adapter 00155DFA9905         7788990 {192.168.50.1}
CustomerA-VM1     Network Adapter 00155DFA9904          445566 {192.168.50.1}
APPController-167 Network Adapter 00155D0F2724               0 {}

Note: you can reset your VLAN Assignment by assigning the machine to VLAN 0.

so to use Windows Network Virtualization we need to  Enable the WNV binding on each host external switch that network traffic pass through.

use the command  Get-VMSwitch -SwitchType External to get a list of the external switches

Get-VMSwitch -SwitchType External

Name               SwitchType NetAdapterInterfaceDescription
—-               ———- ——————————
LAN Virtual Switch External   Intel(R) 82579LM Gigabit Network Connection

To Enable the WNV binding use:

Enable-NetAdapterBinding -InterfaceDescription “Intel(R) 82579LM Gigabit Network Connection” -ComponentID “ms_netwnv”

to check  ms_netwnv Use the command:

Get-NetAdapterBinding -ComponentID ms_netwnv

On Host2 we issue the same command to Enable MS Network virtualization:

Enable-NetAdapterBinding -InterfaceDescription “Realtek PCIe GBE Family Controller” -ComponentID “ms_netwnv

3. Assign Provider IPs

Now we need to define the Provider Address at local host:

Get-NetAdapter –InterfaceDescription “Intel(R) 82579LM Gigabit Network Connection”

Name                      InterfaceDescription                    ifIndex Status       MacAddress             LinkSpeed
—-                      ——————–                    ——- ——       ———-             ———
Ethernet                  Intel(R) 82579LM Gigabit Network Con…      12 Up           38-60-77-CD-80-1E         1 Gbps

New-NetVirtualizationProviderAddress -InterfaceIndex 12 –ProviderAddress 192.168.250.153 -PrefixLength 24

ProviderAddress : 192.168.250.153
InterfaceIndex  : 12
PrefixLength    : 24
VlanID          : 0
AddressState    : Preferred

The Same on Host 2:

New-NetVirtualizationProviderAddress -InterfaceIndex 13 –ProviderAddress 192.168.250.166 -PrefixLength 24

ProviderAddress : 192.168.250.166
InterfaceIndex  : 13
PrefixLength    : 24
VlanID          : 0
AddressState    : Preferred

4. Create Domain and Define Customer Routes

For Hyper-v Hosts to understand that traffic from VMs belong to the same Customer and to provide isolation between customers traffic Hyper-v provide what called routing domain , you can think of routing domains as a physical switch where all customer VLANS live. The Routing domain is built using GUID , so to create to new routing domains use the following:

$CustAGUID = [system.guid]::newguid()
$CustBGUID = [system.guid]::newguid()

# Format the GUID string properly
$CustAGUID = “{” + [string]$CustAGUID + “}”
$CustBGUID = “{” + [string]$CustBGUID + “}”

we will Define a Customer Route for each customer so on host 1:

New-NetVirtualizationCustomerRoute -RoutingDomainID $CustAGUID  -VirtualSubnetID 445566 -DestinationPrefix “192.168.50.0/24“ -NextHop 0.0.0.0 -Metric 255
New-NetVirtualizationCustomerRoute -RoutingDomainID $CustBGUID -VirtualSubnetID 7788990 -DestinationPrefix “192.168.50.0/24“ -NextHop 0.0.0.0 -Metric 255

RESULT Should Show:

RoutingDomainID   : {0501F8D4-3C91-4566-B74B-6C1CAC82C6F7}
VirtualSubnetID   : 7788990
DestinationPrefix : 192.168.50.0/24
NextHop           : 0.0.0.0
Metric            : 255

RoutingDomainID   : {1ABB06F2-CE16-4125-A7E6-510A1B7273DD}
VirtualSubnetID   : 445566
DestinationPrefix : 192.168.50.0/24
NextHop           : 0.0.0.0
Metric            : 255

on Host2:

$CustAGUID=”{1ABB06F2-CE16-4125-A7E6-510A1B7273DD}”
$CustBGUID=”{0501F8D4-3C91-4566-B74B-6C1CAC82C6F7}”

New-NetVirtualizationCustomerRoute -RoutingDomainID $CustAGUID  -VirtualSubnetID 445566 -DestinationPrefix “192.168.50.0/24“ -NextHop 0.0.0.0 -Metric 255
New-NetVirtualizationCustomerRoute -RoutingDomainID $CustBGUID -VirtualSubnetID 7788990 -DestinationPrefix “192.168.50.0/24“ -NextHop 0.0.0.0 -Metric 255

RESULT Should Show:

RoutingDomainID   : {0501F8D4-3C91-4566-B74B-6C1CAC82C6F7}
VirtualSubnetID   : 7788990
DestinationPrefix : 192.168.50.0/24
NextHop           : 0.0.0.0
Metric            : 255

RoutingDomainID   : {1ABB06F2-CE16-4125-A7E6-510A1B7273DD}
VirtualSubnetID   : 445566
DestinationPrefix : 192.168.50.0/24
NextHop           : 0.0.0.0
Metric            : 255

5. Build Lookup table (Routing table) 
The Routing table need to be constructed on both hosts:

$CustAGUID=”{1ABB06F2-CE16-4125-A7E6-510A1B7273DD}”
$CustBGUID=”{0501F8D4-3C91-4566-B74B-6C1CAC82C6F7}”

New-NetVirtualizationLookupRecord -VMName CustomerA-VM1 -VirtualSubnetID 445566 -CustomerAddress 192.168.50.1 -MACAddress 00155DFA9904 -ProviderAddress 192.168.250.153 -Rule TranslationMethodEncap -CustomerID $CustAGUID

New-NetVirtualizationLookupRecord -VMName CustomerA-VM2  -VirtualSubnetID 445566 -CustomerAddress 192.168.50.2 -MACAddress 00155DFAA600 -ProviderAddress 192.168.250.166 -Rule TranslationMethodEncap -CustomerID $CustAGUID

New-NetVirtualizationLookupRecord -VMName CustomerB-VM1 -VirtualSubnetID 7788990 -CustomerAddress 192.168.50.1 -MACAddress 00155DFA9905 -ProviderAddress 192.168.250.153 -Rule TranslationMethodEncap -CustomerID $CustBGUID
New-NetVirtualizationLookupRecord -VMName CustomerB-VM2 -VirtualSubnetID 7788990 -CustomerAddress 192.168.50.2 -MACAddress 00155DFAA601 -ProviderAddress 192.168.250.166 -Rule TranslationMethodEncap -CustomerID $CustBGUID

Managing such configuration for large number of VMs  and host would be a nightmare for administrators , SCVMM 2012 SP1 take care of all these configuration and updates to the lookup Records when machines move between hosts.

Windows Server® 2012 Hyper-V Network Virtualization Survival Guide

BrianEh I.T. Proctology blog

Advertisements

About Hikmat Kanaan

I’m big fan of technology; I have worked almost with every MS windows OS up to Windows 8 and server 2012 including OS deployment, AD and almost every MS OS service included major MS products ISA ,TMS, Exchange, System center ,Sharepoint ,SQL , Storage system, Networking, security, Cisco, HP, and Checkpoint products. Designing and Architecting IT solutions and infrastructure . I do admire automation and working based on best practices toward building highly reliable solution that provide the required services to Business. I also run the Jordan IT professionals user Group http://www.jitpros.net
This entry was posted in Hyper-v, Networking, Private cloud, Windows 2012 and tagged , , , . Bookmark the permalink.

6 Responses to Windows 2012 Hyper-v 3.0 Network Virtualization

  1. sameer says:

    HI… ARTICLE IS NICE.. BUT YOUR NUMBERING IS WRONG.. AFTER 2 YOU PUT 4.

  2. sameer says:

    sorry to point another mistake… on step 4. destination prefix is given as 192.168.1.0/24 instead of 192.168.50.0/24. Please check.

  3. Piotr says:

    Hi, I’v followed your article to implement network virtualization through our HyperV Cluster. But I’m looking for an information that I couldn’t find. What will be the default Gateway for each isolated VM ? Your article, like few others, deals with vm isolation through a 2 node cluster. But what if cistomers VM must communicate out of HyperV such as internet or remote site through IPSEC ?
    Thanks in advance for your lights 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s