Site-to-Site VPN with Wireguard and Docker

Most people interact with one or two networks on a daily basis. The defaults work for watching Netflix, checking your email or catching up on cute Instagram accounts 🐕🐈.

If you manage multiple networks, though, it can start to get tricky to access internal resources across different internal networks. Maybe you want to login to a file server at a second house. Maybe you need to change some Wi-Fi settings to troubleshoot an issue with a coworker at another office. Maybe your mom is having trouble printing.

A site-to-site network extends a private network across multiple places. So if you have more than home you can still access the stuff at Home A from Home B and vice versa. The best part is you can do it without putting that stuff on the public Internet — keeping your resources safe and secure on your extended private network. And, because it happens at the network level, you also don’t need to worry about connecting / disconnecting to a client VPN on each device.

Cool! So what’s the catch? Setting up these types of networks can be tedious, and is usually the work of network engineers. WireGuard offers an alternative to more traditional tools like IPSec or OpenVPN that makes setting up a site-to-site VPN much more simple. It also employs strong cryptography and is kernel based (making it insanely fast).

An Example Network

So here’s what a simple, multiple site scenario might look like before setting up a site-to-site VPN. If you want to access private resources (like security cameras), you might need to poke holes in the firewall with port forwarding for each server, or setup a bastion SSH server.

HomeCabinServer
Local resources:NAS for storage and media
Network security cameras
Home automation
Printer / scanner Network TV receiver
NAS for backup and media
Network security cameras
Network weather station
Nature webcam
Public websites
Internal websites
Type:SiteSiteSingle server
Private IP:10.100.0.5/3210.200.0.5/3210.300.0.1/32
Private network:10.100.0.0/1610.200.0.0/1610.300.0.0/16
Public address:252.193.48.64/3294.43.15.199/32193.111.10.3/32
Domain:home.mycoolnetwork.netcabin.mycoolnetwork.netweb.mycoolnetwork.net
WireGuard IP:
(ignore for now)
192.168.99.1/32192.168.99.2/32192.168.99.3/32

Simple Approach

There are an infinite number of ways to setup, customize and secure your site-to-site network with WireGuard. I’m going to share an extremely simplified way to setup a mesh like network between two full networks and a single server (public cloud VM) using containers. Why? Because it’s super easy.

This approach uses docker-compose to pull images, grant necessary system capabilities and handle networking and auto start. Within the container it also uses the wg-quick feature of WireGuard to setup the barebones routes needed for the peers to communicate with each other.

It also uses wireguard-go which is a version of WireGuard that will work without installing customer kernel modules (making it much more container friendly!). This could impact the performance of a VPN setup, but will work if you have peers on a modern OpenVZ host, for example. Notably, if your host was wireguard installed already you can use it directly.

Prerequisites

You’ll need a few things to get started, and I’ll assume you have these ready to go.

  1. Computer or Virtual Machine (VM) running Docker (at each site): WireGuard runs on just about any OS, many routers and even Raspberry Pi. I’ve set it up on all of the above, but I’ve found the easiest solution is just to spin up an Ubuntu VM and install Docker.
  2. Public IP address at each site: this is what will enable the “mesh” feature of your network. If any one site goes offline (power outage, ISP failure, etc) the rest of the network will keep working. Technically, only site has to have a public IP and it can act as a hub.
  3. Network subnets that don’t collide: you’ll notice in my example of the the networks are in their own, non colliding /16 subnet. This means you don’t need to do any funny business to translate the address and makes thing much more simple. It isn’t strictly necessary, but I highly recommend.

Configuration

Each site will require two configuration files. In this section we’ll look at docker-compose.yml (generally the same for all peers) and wg0.conf (different for each peer).

Docker: docker-compose.yml

Pick a sane directory on your WireGuard peers like /containers/wireguard. In that folder you can copy the below docker-compose file to /containers/wireguard/docker-compose.yml. You should also create a /containers/wireguard/config directory, too.

version: '3.3'
services:
  wireguard:
    image: masipcat/wireguard-go:latest
    cap_add:
     - NET_ADMIN
    sysctls:
     - net.ipv4.ip_forward=1
    volumes:
     # This is what lets us create a wg interface without kernel module on host 
     - /dev/net/tun:/dev/net/tun
     # Folder with 'publickey', 'privatekey' and 'wg0.conf'
     - ./config:/etc/wireguard
    environment:
     - WG_COLOR_MODE=always
     - LOG_LEVEL=info
    network_mode: host
    restart: always

You can use the same file on each of your peers (2, 200, it doesn’t really matter). We don’t want to start the container yet so move on to the next step.

Why host networking? Isn’t this a containerization sin? Yeah, it’s not ideal. Because this container will be providing network level services, and creating / modifying routes at the host level, we are giving it host networking access and administrator capabilities. There are arguably better ways to configure this, but they are more complicated and out of scope for this post.

WireGuard: wg0.conf

This is the file that WireGuard (and its included wg-quick tool) will use to setup the tunnelled interface and configure our network. Each one will be slightly different. You can either use a tool to generate and update these automatically, or can create them manually.

If you just have two or three sites it would be easy to hand bomb the config files. In my case I have 6 sites so I decided to use a useful Python based configuration generator. I like it because I can export my mesh network as JSON, edit, re-import, and regenerate config files as needed.

I’ve created a little docker image with the prerequisites installed so you can generate the config files without having to install WireGuard or dependancies. When it starts type “help” for usage details.

docker run -it -v "$(pwd)/configs:/configurator/configs" quacktacular/wireguard-mesh-configurator

One annoyance is that the AddPeer command only accepts one address CIDR per peer. As you’ll see in the configs we’ll be using at least two (I’ll explain why in a minute). My suggestion would be to add with one address in the tool, then just JSONSaveProfile and edit the address string in the JSON manually (it only validates when you add with tool). I plan to make a simpler Python tool to load config from YAML soon.

The finished profile should end up something like this. The address field should include the full CIDR that you want to be able to communicate with your remote networks, the private IP (if it isn’t already assigned to an interface on the peer), and WireGuard IP. I’ll explain that now.

{
    "peers": [
        {
            "address": "10.100.0.5/32,10.100.0.0/16,192.168.99.1/32",
            "public_address": "home.mycoolnetwork.net",
            "listen_port": "51820",
            "private_key": "CG0svjlK7NdZ3U0MdYQzBHx7adDi1p2UlhPFXdH4HHw=",
            "keep_alive": true,
            "preshared_key": null,
            "alias": "",
            "description": ""
        },
        {
            "address": "10.200.0.5/32,10.200.0.0/16,192.168.99.2/32",
            "public_address": "cabin.mycoolnetwork.net",
            "listen_port": "51820",
            "private_key": "0AvTiKBleIr73VLj+AxaiVJEn+OIUPIa9Fw05GW4bUY=",
            "keep_alive": true,
            "preshared_key": null,
            "alias": "",
            "description": ""
        },
        {
            "address": "10.300.0.1/32,192.168.99.3/32",
            "public_address": "web.mycoolnetwork.net",
            "listen_port": "51820",
            "private_key": "oPzz/yB7CUbYsFCCJUsGPjnEYrfrlDRq+whn0y7/Znw=",
            "keep_alive": true,
            "preshared_key": null,
            "alias": "",
            "description": ""
        }
    ]
}

Ok! So now you can GenerateConfigs ./ or write them manually. Following our example you’ll end up with three config files.

10.100.0.5.conf

[Interface]
PrivateKey = CG0svjlK7NdZ3U0MdYQzBHx7adDi1p2UlhPFXdH4HHw=
Address = 10.100.0.5/32,10.100.0.0/16,192.168.99.1/32
ListenPort = 51820

[Peer]
PublicKey = Jkdn621+amuCV8Wj7YQLMydtE9GO5kpq+oZdK/17XAY=
AllowedIPs = 10.200.0.5/32,10.200.0.0/16,192.168.99.2/32
Endpoint = cabin.mycoolnetwork.net:51820
PersistentKeepalive = 25

[Peer]
PublicKey = VaVaSY6SkizEhexj9vSTkzKgaIo5MwMnulu6I/D+iAI=
AllowedIPs = 10.300.0.1/32,192.168.99.3/32
Endpoint = web.mycoolnetwork.net:51820
PersistentKeepalive = 25

10.200.0.5.conf

[Interface]
PrivateKey = 0AvTiKBleIr73VLj+AxaiVJEn+OIUPIa9Fw05GW4bUY=
Address = 10.200.0.5/32,10.200.0.0/16,192.168.99.2/32
ListenPort = 51820

[Peer]
PublicKey = rF+ewW+GeoIqgsoyzeFpjBOuACoN7u6gTNIgP6Bq0Rw=
AllowedIPs = 10.100.0.5/32,10.100.0.0/16,192.168.99.1/32
Endpoint = home.mycoolnetwork.net:
PersistentKeepalive = 25

[Peer]
PublicKey = VaVaSY6SkizEhexj9vSTkzKgaIo5MwMnulu6I/D+iAI=
AllowedIPs = 10.300.0.1/32,192.168.99.3/32
Endpoint = web.mycoolnetwork.net:
PersistentKeepalive = 25

10.300.0.1.conf

[Interface]
PrivateKey = oPzz/yB7CUbYsFCCJUsGPjnEYrfrlDRq+whn0y7/Znw=
Address = 10.300.0.1/32,192.168.99.3/32
ListenPort = 51820

[Peer]
PublicKey = rF+ewW+GeoIqgsoyzeFpjBOuACoN7u6gTNIgP6Bq0Rw=
AllowedIPs = 10.100.0.5/32,10.100.0.0/16,192.168.99.1/32
Endpoint = home.mycoolnetwork.net:
PersistentKeepalive = 25

[Peer]
PublicKey = Jkdn621+amuCV8Wj7YQLMydtE9GO5kpq+oZdK/17XAY=
AllowedIPs = 10.200.0.5/32,10.200.0.0/16,192.168.99.2/32
Endpoint = cabin.mycoolnetwork.net:
PersistentKeepalive = 25

Copy the text of each of these to their respective peer at /containers/wireguard/config/wg0.conf.

Start Containers

Now that all the configuration is in place you can docker-compose up -d in /containers/wireguard/ on each peer. The container will use wg-quick to create the necessary interfaces and routes.

After the image is pulled and container started, you should be able to ping between the peers on the private and WireGuard IP address.

Routing

Once you have everything working between your peers, you still have some work to do. You’ll want to login to your router and setup static routes for each remote site. Each rule should direct traffic towards the remote subnets (10.300.0.0/16 for example) to the private IP of local peer (Docker VM, computer) as a next hop.

Troubleshooting

Ok so there is a pretty good chance something will go sideways the first time you try this. Finding the issues is usually pretty easy though.

  • wg show: You can exec into the container docker exec -it wireguard_wireguard_1 /bin/sh on the peer and run wg show. It will let you know if the peers can communicate (handshake == good)
  • tcpdump: You can run tcpdump -i wg0 on the remote server, then on your local machine ping: ping 10.200.0.5
  • traceroute: You can run this command to find the path between your local machine and the other peer

Leave a Reply

Your email address will not be published. Required fields are marked *