Portable SD-WAN Over Private Networks

SD-WAN is about Internet access right? No not always…

If you were to listen to most marketing information about SD-WAN you would think that it is designed to work purely on public internet links as that is probably the easiest application to explain, and just about anyone can understand the benefits of using multiple internet links of different types from different ISPs to provide resilient internet and site to site VPN connectivity.

Private networks can be used for SD-WAN too.

However the ‘WAN’ in SD-WAN can be just about any type of connectivity – private links included, and depending on your your requirements you might only want to use private links.

Yesterday Andy Harris and I did a technology demo for a customer where the primary WAN connectivity was actually a local WiFi infrastructure network and I’m going to share some detail on that to show you why that can be really useful.

Resilient local network connectivity

The customer needed to demonstrate a man portable battery powered SD-WAN solution as part of a larger opportunity where a smartphone/tablet based application needed to be run reliably across a golf course.

The key factor was the local network reliability and resilience as the application was very time sensitive. The idea was that although a wireless network was going to be available that could be used as a primary connection they also wanted to be able to use alternative network connectivity if needed in case the local wireless network became unreliable – and they wanted to be able to extend the wireless network coverage on demand.

I was asked to put together a solution design to fulfil these requirements and then demonstrate and prove the network resilience to the end customer. An additional challenge was that the location chosen for the demo didn’t actually have a wireless network that covered the course currently so I needed to bring one with me – one that could cover three holes of the course to allow for a ‘real world’ demo.

Initial Design Concept

Although it looks complicated, the idea was relatively simple.

1. Set up a base station location and provide it with internet access using whatever connectivity was desired (eg LTE, Fixed line Internet and satellite).
2. Build a battery powered bag and fill it with as much connectivity as possible (2.4 + 5Ghz Wireless & LTE)
3. Set up some temporary WiFi infrastructure to cover the course
4. Use SpeedFusion to link the bag and the base station by combining all available links.

WiFi Everywhere

As you can see, my plan was to not only deploy a temporary 5Ghz Wireless Network to cover the course, but also mix in 2.4Ghz WiFi connectivity wherever I could as well. The idea being that the 2.4Ghz WiFi network wouldn’t interfere with the 5Ghz network but could act as a way to extend the coverage to tricker to reach locations – especially since the bags themselves could act as 2.4Ghz network repeaters.

Network Build – Base Station

This was actually the easiest bit of the build since helping customers get internet connectivity into random temporary locations is what I do all the time. For this technology demo I knew that there was great EE LTE coverage at the location so I decided to rely on that for general internet connectivity and mixed in a Three LTE SIM as well to provide some mobile network operator resilience too just in case.

Since we were only planning on building two mobile bags I used a HD2 at the base station. The HD supports 2 active cellular connections, has two wired WANs and can use WiFi as a WAN as well. That capability combined with its support for two remote SpeedFusion Peers made it a perfect fit.

Network Build – Bags

I also used a HD2 in the bags as I wanted to use SpeedFusion bonding for network resilience. Each HD2 had the following network connections:

1. CELL1 : LTE EE SIM
2. WAN1: 5Ghz WiFI Infrastructure
3. WAN2: 2.4Ghz Mesh Network
4. WIFI WAN: 2.4Ghz Infrastructure

Network Build – 5Ghz WiFi Infrastructure

We had to build a temporary rapidly deployable WIFI network as part of the technology demo as the test location didn’t have one. Considering the area we needed to cover I decided to use Ubiquiti Infrastructure equipment for this job as I had witnessed first hand the range their M5 Rockets and Omni Antennas could achieve on the Realm Pictures Live FPS deployments.

Although I work with and deploy wireless networks regularly – including long range point to point links, I don’t have the opportunity to do as much ‘green space’ WiFi as I would like, so to play safe I used the ubiquiti kit I know best for this demo as I knew it would work – even if it was a little overkill for the job.

To that end I deployed M5 Rockets on Tripods with a mix of omnidirectional and 60 degree sector antennas just to make sure the area of the course reserved for the demo would be covered. The deployment looked like this:

The antennas were rigged onto a quick deploy tubular sleeve that we slipped over some light weight tripods. These enabled us to keep the deployment time for the whole wifi network down to 30mins as we didn’t need any tools to bolt the antennas onto the tripods. Here is a photo of the last tripod with its sector and omni antennas installed. You can see we used sandbags to provide some stability and the whole tripod is powered by a battery and a pair of POE power injectors (just visible in the blue plastic bag at the bottom of the tripod).

With the 5Ghz Network deployed it was time to put the 2.4Ghz network in place.

Network Build – 2.4Ghz WiFi Infrastructure

The 2.4Ghz Wifi infrastructure was not intended to provide full cover of the demo area – rather it was there to show seamless failover between the 5 & 2.4Ghz wifi networks worked as they became available.

To demo this I had an AP One AC Mini powered on near the base station with an SSID of SF-WIFI and both the base station and Bag 1 HD2 had a WiFi as WAN profile set up so they could connect and use it when in range.

When both devices had an active connection to the 2.4Ghz wifi network they could then directly communicate with each other and so were able to build a SpeedFusion tunnel across the 2.4Ghz network as an additional path for network resilience.

Network Build – 2.4Ghz WiFi Mesh

When I sat down to design the topology for this test I realised that it would be really useful for a bag to be able to connect to the network via another bag when out of range of the infrastructure WifI. This would then allow connectivity ‘around corners’ and over hills etc that would otherwise be out of coverage.

I thought that was going to be an easy thing to achieve but there are a surprising number of complications that arise. First of all I attempted to configure an SSID on BAG1 called MESH and then set BAG2 to connect to it using its WiFI as WAN. This did work but I quickly realised that things were going to get messy if BAG1’s wifi WAN was also looking for the same SSID.

So I did what all good engineers should do and asked someone who knows more about Peplink Wifi capabilities than me – the Peplink core Wifi development team. They came up with a really elegant way to achieve this and within a couple of hours had churned out a beta firmware for me to test. It worked brilliantly and I plan to do a dedicated blog post on this at a later date.

The end result was that the HD2’s could route traffic through each other, so when a BAG was out of range of the infrastructure WiFi networks but in range of another bag that was still connected they could piggy back that bags connectivity and still access the internet and send traffic back to the base station.

The Four states of Connectivity

The Network topology allowed for four demonstrable connectivity states as part of the test day with seamless failover between each of them as the available connectivity at a bag changed depending on its location.

1. Multi-WiFi WAN – 2.4 & 4Ghz Wifi used
2. General Cover – 5Ghz Wifi only
3. Range Extender – One Bag connected via another
4. Cellular Failover – If no wifi is available at all failover to cellular.

The Bags – Hyper Connected Rucksacks with a Twist

When it came to the portable SD-WAN bags we were planning just to pop to Argos and get something normal then I found the Paxis Pak and ‘normal’ just wasn’t going to cut it any more.

These bags look like any normal, comfy well built bag should at first glance – but they have a party trick. When you pull the toggle on the shoulder strap the whole bottom section swings round to the front and becomes easily accessible.

That bottom section was an obvious place to put the batteries and USB charger for our phones and tablets, and perhaps less obviously the swing arm was also a great place to rest a laptop when running network tests as well as a way to stand the rucksack up on the ground.

With the HD2 inside the bag, batteries installed in the swing arm, and a Ubiquiti bullet AP with its omni antenna strapped to the outside, we had a unique and versatile portable SD-WAN solution.

Putting it all together and running the tests

Once all the connectivity was in place I needed to configure SpeedFusion VPN between the bags and the base station to allow for seamless failover between the WAN connections in the Bags as they moved in and out of wireless network coverage.

This would have been really easy if I had used SIMs with public IP addressing as the bags would have been able to route traffic directly to each other over cellular just like they could over the local WiFi – but the SIMs I had were all dynamically addressed by the mobile network operator.

Luckily I have my handy Fusionhub virtual appliance hosted in the cloud so I used this to enable the base station and bags to route traffic to each other. The end result was actually two VPN tunnels for each HD2 in the Bags. One using just the local wifi services terminated directly point to point with the base station HD2. The other used just the cellular connections and terminated to the publically hosted FusionHub. I then set the HD2s in the bags to failover between the VPN connections in precedence with the one using local wifi set as primary and the cellular as secondary.

The end result was seamless roaming for the bags across the entire test area and beyond. The tests went really well and the customer was impressed by the ability to roam between any available connectivity on site whilst maintaining a resilient connection to the base station.

Finally – Remote Management and Monitoring in inControl 2

One of the most powerful parts of the demo was when we showed the customer InControl 2’s capabilities to monitor and track the bags as they roamed across the course. This proved that we had maintained connectivity throughout the test period. Below is the GPS track for BAG 2.
 

Is this actually SD-LAN?

As I’m writing this post and thinking about what we achieved here – this use of SD-WAN technologies on private local area network connections, I’ve been asking myself if this should actually be called SD-LAN (ie SDN) instead of SD-WAN.

My conclusion is that it shouldn’t.

SD-WAN technologies exist and are deployed on the basis that the connections used are expected to be variable and at times unreliable. The normal cause of this is that WAN connections  typically use the internet as a connection path (which can get congested) using physical connection technologies that can fail due to environmental reasons (like builders digging through cables).

In this deployment above, SD-WAN is being used for exactly the same reasons – and its only by intelligently monitoring, measuring and combining all available connection types (WiFi and Cellular) which in themselves can get congested and be affected by environmental conditions so become unreliable, that we can achieve the desired reliability.

This is SD-WAN, just a different application of the technology and a useful one at that.