Configuring VXLAN and GRE Tunnels on OpenvSwitch
2013-05-05 08:42
453 查看
CONFIGURING VXLAN AND GRE TUNNELS ON OPENVSWITCH
UPDATE: VXLAN ENCAPSULATION
VXLan is now upstreamed into the Master build. It is worth mentioning that is the encapsulation only, not the Multicast control plane functionality. That is being worked on by Mestery currently. So basically no need to pull from the fork but pull directly from
the Open vSwitch Master. ‘git clone git://openvswitch.org/openvswitch’ have the VXLan encap included.
I have done a couple of GRE tunnel how-tos using OpenvSwitch (OVS). I ran across the git tree of the VXlan implementation which is still in development I am sure. I don’t want anyone to think this is anything other than me seeing the tree on GitHub. I had been
itching to give VXLan a spin in OVS so why not ferret out someones tree on GitHub. I believe VXLan is still scheduled to officially release soon in OpenvSwitch. So here are the steps for installing, configuring tunnels on OpenvSwitch with VXLan and GRE encapsulations.
At the end we will compare some of the protocols with difference MTU sizes. The results were interesting I think (for a nerd). We will be installing and then configure both GRE and VXLan encapsulated tunnels using Open vSwitch.
I like seeing some collaboration of really smart people from different companies as displayed in the GPL below.
/* * Copyright (c) 2011 Nicira Networks.
* Copyright (c) 2012 Cisco Systems Inc.
* Distributed under the terms of the GNU GPL version 2.
* * Significant portions of this file may be copied from parts of the Linux
* kernel, by Linus Torvalds and others. */
Figure 1. Example of how tunnels can be leveraged as overlays.
By the way I should probably disclaim now that huge Layer2 networks do not scale and huge Layer3 networks do. Host count in a broadcast domain/Vlan/network should be kept to a reasonable 3 digit number. Cisco is quick to point out that OTV is the solution over
WAN’s for extending Layer 2 networks with OTV as the solution to extend Layer 2 Vlans. That said Overlays
are flat out required to overcome Vlan number limitations and have lots of potential with programmatic orchestration.
Figure 2. OVS punts the first packet to user land for the forwarding decision and passes the data path back to the data path in the kernel for subsequent packets in the flow. slow path for the first packet then fast path for the rest.
I do some simple Iperf tests at the end of the post with 1500 and 9000 byte MTUs. The numbers are kind of fun to look at from the result. For a really nice analysis take a look at Martin’s post at
Network Heresy on comparing STT, Linux Bridging and GRE. I am going to spin up some VMs on the VXLan tunnel later this week and measure the speeds a little closer and see how GRE and VXL
4000
an stack up to one another from hosts using overlays. I just ran some
Iperfs from the hypervisor itself rather than VMs. OVS also supports Capwap encapsulation that performs mac in GRE which is rather slick. Wonder why we do not hear much about that. I am going to dig in when I get back to home from the road later next week.
Figure 3. Lab Setup basically has a fake interface up with Br1. Real world Br1 would have VMs tapped on it. The video uses br1 and br2 but it got to be confusing for people so I changed it to br0 and br1 to match most peoples eth0
= br0 NIC naming in ifconfig.
Quick Video of the fast install below.
For those familiar with the build you can just paste the following in your bash shell as root. To walk through the install skip
the following snippet.
1234567891011121314151617181920 | apt-get updateapt-get install -y git python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config linux-headers-`uname -r`git clone git://openvswitch.org/openvswitchcd openvswitch./boot.sh./configure --with-linux=/lib/modules/`uname -r`/buildmake && make installinsmod datapath/linux/openvswitch.kotouch /usr/local/etc/ovs-vswitchd.confmkdir -p /usr/local/etc/openvswitchovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschemaovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-fileovs-vsctl --no-wait initovs-vswitchd --pidfile --detachovs-vsctl show |
Install dependencies
1 2 | ; html-script: false ]apt-get update apt-get install python-simplejson python-qt4 python-twisted-conch automake autoconf gcc uml-utilities libtool build-essential git pkg-config |
12 | git clone git://openvswitch.org/openvswitchcd openvswitch |
1 2 3 4 5 6 | ; html-script: false ]./boot.sh ./configure --with-linux=/lib/modules/`uname -r`/build make make install #Load the OVS Kernel Module insmod datapath/linux/openvswitch.ko |
1234 | ; html-script: false ]#If you are running a stripped version of Nix like an EC2 cloud image and get an error along the lines of this:configure: error: source dir /lib/modules/3.2.0-23-virtual/build doesn't exist#Pull down the headers for your kernel. sudo apt-get install linux-headers-`uname -r` |
1 2 3 | ; html-script: false ]touch /usr/local/etc/ovs-vswitchd.conf mkdir -p /usr/local/etc/openvswitch ovsdb-tool create /usr/local/etc/openvswitch/conf.db vswitchd/vswitch.ovsschema |
Start ovsdb-server, this stores the config into a file that is persistent even after restarts.
123456 | ; html-script: false ]ovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file |
1 2 3 4 5 6 7 8 | ; html-script: false ]#Only need to run this the first time. ovs-vsctl --no-wait init #Start vswitch ovs-vswitchd --pidfile --detach #Verify the kernel module(s) in case you didn't earlier and get errors. #lsmod | grep br #brcompat 13512 0 #openvswitch 98196 1 brcompat |
1 | insmod datapath/linux/openvswitch.ko |
1 2 3 4 5 | ; html-script: false ]ovs-vsctl show b6d574d7-5582-4cc0-93e5-a90cf0eb0a38 root@demo-139:/home/ubuntu/ovs-vxlan# ovs-vsctl --version ovs-vsctl (Open vSwitch) 1.8.90 Compiled Aug 19 2012 06:23:36 |
CONFIGURE LINUX NETWORKING
I have one NIC (eth0) on the same LAN segment/network/vlan.
We are attaching eth0 to br1 and applying an IP to the bridge interface.
We are attaching an IP to br1. br1 is the island that we are building a tunnel for hosts to connect on. Without the VXLAN tunnel, the two br1 interfaces should not be able to ping one another. Note: This is being setup on the same subnet, it is important to
keep in mind that the VXLAN framing will allow for the tunnel to be established over disparate networks. E.g. can be done over the Internet etc.
Configuration for Host 1
12345678 | ; html-script: false ]#Host 1 Configurationovs-vsctl add-br br0ovs-vsctl add-br br1ovs-vsctl add-port br0 eth0ifconfig eth0 0 && ifconfig br0 192.168.1.10 netmask 255.255.255.0route add default gw 192.168.1.1 br0ifconfig br1 10.1.2.10 netmask 255.255.255.0ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11 |
1 2 3 4 5 6 7 8 | ; html-script: false ]#Host 1 Configuration ovs-vsctl add-br br0 ovs-vsctl add-br br1 ovs-vsctl add-port br0 eth0 ifconfig eth0 0 && ifconfig br0 192.168.1.11 netmask 255.255.255.0 route add default gw 192.168.1.1 br0 ifconfig br1 10.1.2.11 netmask 255.255.255.0 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.10 |
Your Linux routing table and ifconfig should now look something like this:
1234567891011121314151617181920212223242526272829303132333435363738394041 | root@ub64:/home/brent/openvswitch# route -nKernel IP routing tableDestination Gateway Genmask Flags Metric Ref Use Iface0.0.0.0 192.168.1.1 0.0.0.0 UG 0 0 0 br010.1.2.0 0.0.0.0 255.255.255.0 U 0 0 0 br1192.168.1.0 0.0.0.0 255.255.255.0 U 0 0 0 br0root@ub64:/home/brent/openvswitch# ifconfigbr0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet addr:192.168.1.10 Bcast:192.168.1.255 Mask:255.255.255.0 inet6 addr: fe80::285f:3ff:fea2:dafc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:622 errors:0 dropped:0 overruns:0 frame:0 TX packets:54 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:171756 (171.7 KB) TX bytes:3832 (3.8 KB) br1 Link encap:Ethernet HWaddr ea:34:7d:49:e7:49 inet addr:10.1.2.10 Bcast:10.1.2.255 Mask:255.255.255.0 inet6 addr: fe80::c0a7:aaff:fe90:29dc/64 Scope:Link UP BROADCAST RUNNING MTU:1500 Metric:1 RX packets:12 errors:0 dropped:0 overruns:0 frame:0 TX packets:13 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:896 (896.0 B) TX bytes:986 (986.0 B) eth0 Link encap:Ethernet HWaddr 00:0c:29:65:fd:82 inet6 addr: fe80::20c:29ff:fe65:fd82/64 Scope:Link UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 RX packets:1222493 errors:0 dropped:0 overruns:0 frame:0 TX packets:53824 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:653399936 (653.3 MB) TX bytes:4089412 (4.0 MB) lo Link encap:Local Loopback inet addr:127.0.0.1 Mask:255.0.0.0 inet6 addr: ::1/128 Scope:Host UP LOOPBACK RUNNING MTU:16436 Metric:1 RX packets:7088 errors:0 dropped:0 overruns:0 frame:0 TX packets:7088 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:0 RX bytes:537609 (537.6 KB) TX bytes:537609 (537.6 KB) |
1 2 3 4 5 6 7 8 9 10 | ; html-script: false ]Aug 19 06:34:04 demo-139 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl del-br br1 Aug 19 06:34:04 demo-139 ovs-vswitchd: 00026|dpif|WARN|failed to enumerate system datapaths: No such file or directory Aug 19 06:34:04 demo-139 ovs-vswitchd: 00027|bridge|INFO|ovs-vswitchd (Open vSwitch) 1.8.90 Aug 19 06:34:20 demo-139 ovs-vsctl: 00001|vsctl|INFO|Called as ovs-vsctl add-br br1 Aug 19 06:34:20 demo-139 ovs-vswitchd: 00028|dpif|WARN|failed to enumerate system datapaths: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00029|dpif|WARN|failed to create datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00030|ofproto_dpif|ERR|failed to open datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00031|ofproto|ERR|failed to open datapath br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00032|bridge|ERR|failed to create bridge br1: No such file or directory Aug 19 06:34:20 demo-139 ovs-vswitchd: 00033|bridge|INFO|ovs-vswitchd (Open vSwitch) 1.8.90 |
12345678910111213141516171819 | ; html-script: false ]#Find the OVS PIDs from PSps -ea | grep ovs16369 ? 00:00:00 ovsdb-server16372 ? 00:00:00 ovs-vswitchd16373 ? 00:00:00 ovs-vswitchd16375 ? 00:00:00 ovs_workq#Terminate themkill 16369kill 16372kill 16375#Then restart the OVS serverovsdb-server /usr/local/etc/openvswitch/conf.db \--remote=punix:/usr/local/var/run/openvswitch/db.sock \--remote=db:Open_vSwitch,manager_options \--private-key=db:SSL,private_key \--certificate=db:SSL,certificate \--bootstrap-ca-cert=db:SSL,ca_cert --pidfile --detach --log-file#Start vswitchd (JSON DB)ovs-vswitchd --pidfile --detach |
1 2 | ; html-script: false ]ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 gre1 -- set interface gre1 type=gre options:remote_ip=192.168.1.11 |
#(Remember your switch needs to have >= your MTU.
Figure 4. Output of GRE Tunnels running with 1500 byte MTUs.
Figure 5. Output of GRE Tunnels running with 9000 byte MTUs.
Now delete the GRE ports
1 | ovs-vsctl del-port gre1 |
1 2 | ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.10 ovs-vsctl add-port br1 vx1 -- set interface vx1 type=vxlan options:remote_ip=192.168.1.11 |
Figure 6 Output of VXLan Tunnels running with 1500 byte MTUs.
相关文章推荐
- 【OpenStack】OpenVswitch GRE MTU problem and solution
- Install Openvswitch by binary and source way on rhe6.3 ( by quqi99 )
- Multiple Private Networks with Open vSwitch GRE Tunnels and Libvirt
- GRE and VXLAN with Open vSwitch
- GRE and VXLAN with Open vSwitch
- What is the "internal" interface and port for on Openvswitch?
- Understanding and Configuring Spanning Tree Protocol on Catalyst Swiches
- [转]Installing and Configuring target iSCSI server on Windows Server 2012
- Install Openstack with Openvswitch Plugin of Quantum on rhel6.3 by RPM Way ( by quqi99 )
- Docker+OpenvSwitch走进VxLAN的世界
- Open Settings on cciPhone and navigate to General -> Device Management, then select your Developer A
- Known issue about configuring BAM Portal and WSS based on different IIS versions
- Using GRE Tunnels with Open vSwitch
- Configuring PATH and other environment variables in a bat file on Windows 7
- Using GRE Tunnels with Open vSwitch
- Mininet and Open vSwitch on CentOS 7
- No statements may be issued when any streaming result sets are open and in use on a given connection
- Using GRE Tunnels with Open vSwitch
- Neutron 理解 (3): Open vSwitch + GRE/VxLAN 组网 [Netruon Open vSwitch + GRE/VxLAN Virutal Network]