systemd, bridges, containers, and IPv6

I apologize for the quickie here; it’s late, I’m tired, and I just cobbled together a solution to a problem that’s been bothering me since this weekend. First, some backstory since I know you guys appreciate it when I go off on tangents. (And if you don’t, you can just scroll passed this nonsense.)

I’ve been migrating most of my services on my home file server into their own containers. I already did this on my VPS, and I plan on doing it to others eventually, because the service isolation is somewhat helpful and prevents stupid mistakes from doing equally stupid things to the host. Containers aren’t a panacea, of course, and the man pages specifically warn against using them as a security apparatus. It’s not effective, so they say, but I’m also a believer in defense-in-depth.

Anyway, the intent has been mostly a matter of isolation. I have a Minecraft server running, among other things, and I’ve been wanting to isolate it from the rest of the system. Not only does this make configuration somewhat easier since I can have service-specific user accounts without polluting the host passwd entries, but it means I can provide some package isolation as well (no Java on the host–yay!). This isn’t without its shortcomings, mind you, and it’s been one Hell of an interesting battle, particularly when IPv6 is thrown into the mix. But I digress, and the server-specific configuration is a welcome post for another day.

In the coming weeks, I’ll see about making a post on using systemd-nspawn for container isolation, why I chose it over Docker and LXC, how to get it working with user namespaces, and how to configure it with IPv6 addresses routed to each container (rather than through auto-config). Hint: you need routable prefixes like a /48 or /56–trying to (ab)use the neighbor discovery proxy and the respective sysctl settings to segment parts of your /64 non-routable prefix won’t work. Don’t even bother, because it’s not worth the misery.

Meanwhile, back to business: Since I do quite a bit of development of various sorts on my desktop, I also wanted to muck about with containers (which I’ve done for isolating builds), but I wanted to bridge the container with my network-facing interface in a manner that it would happily pull down IPv4 and IPv6 addresses from DHCP. My network, on both IP protocols, is serviced by DHCPv4 and DHCPv6 with more-or-less static assignments tossed out to the appropriate systems with a few exceptions. Specifically, connected devices either get their automatically configured IPv6 address, a static address plus an automatically configured address, or some mix (or absence) thereof. Ideally, containers should receive these addresses as well and behave just as an ordinary OS instance or virtual machine would.

Unfortunately, if you follow the systemd-networkd guides for Arch, you’ll quickly wind up in a circumstance where your network is only going to be half-configured. You might be able to get the IPv4 addresses you expect, if you configure the bridge for DHCP, but your IPv6 range isn’t necessarily going to work even if it does accumulate a set of addresses. Bummer.

Oh, and pay attention to casing: If you enter any of the values entirely in lowercase even by accident, nothing will work. I made this mistake, and my eyes never saw anything wrong because I was too focused on reading “bridge” to notice that it should have been entered as “Bridge.” The obsession, in this case, with the word “bridge” apparently served as an override for sensible parsing that included letter case, but this is a fairly common problem when you’ve been steeped in a similar class of issues for days on multiple systems; it’s a similar phenomenon to semantic satiation wherein words begin to look “strange” if you stare at them too long.

For the purposes of this discussion, we’ll assume that we’ve created files in a similar spirit to the Arch guide for systemd-networkd bridges: vbr0.netdev, vbr0.network, and whatever the name is of your hardware interface (mine is enp6s0, so I named its configuration rather creatively: enp6s0.network).

As with the guide, our files are as follows:

# vbr0.netdev
[NetDev]
Name=vbr0
Kind=bridge
# vbr0.network
[Match]
Name=vbr0
 
[Network]
DHCP=yes

(Notice DHCP=yes above. If you need static assignments, change this accordingly.)

And finally:

# enp6s0.network
[Match]
Name=enp6s0
 
[Network]
IPv6AcceptRouterAdvertisements=no
Bridge=vbr0

Oh! What’s this IPv6AcceptRouterAdvertisements=no? More importantly why do we need it? We want to accept IPv6 RAs on our network, don’t we?

The answer might surprise you: Yes and no. Yes, we need router advertisements, but no, we don’t need them on the bridge’s slave device. (Bonus: You won’t find this in the current documentation.) If you fail to add this option, your physical ethernet device will collect an autoconfiguration address, probably accumulate the appropriate route information, configure itself for IPv6, all while your bridge device does the same thing. You’ll be left with an interface that works for IPv4, is configured for both IPv4 and IPv6, but refuses to do anything with its IPv6 assignments. Obviously, if your network isn’t running radvd or similar, you won’t need this, and you certainly won’t need this if you’re not using IPv6. However, if you’re not running IPv6, you probably aren’t reading this article, are you?

If you’ve made it this far, I assume you’re interested in the final moment of truth and the culmination of our journey together. I therefore present to you a systemd-nspawn container with DHCP assignments for IPv4 and IPv6 auto configuration (addresses redacted for privacy):

[root@buildbot network]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: host0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 1e:21:2b:fc:51:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.5.24/23 brd 192.168.5.255 scope global dynamic host0
       valid_lft 938sec preferred_lft 938sec
    inet6 [prefix redacted]:664e:9f3d/128 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 [prefix redacted]:fefc:51b9/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::1c21:2bff:fefc:51b9/64 scope link 
       valid_lft forever preferred_lft forever

If you haven’t bothered planning your IPv6 migration, you really ought to. The inevitable is coming sooner or later. You do want to be prepared, don’t you?

No comments.
***