Hi there! It appears you're visiting my site from an IPv6-enabled address! That's awesome! Your IP address is: 2a01:4f8:192:296::2. Spread the word and encourage IPv6 adoption!

systemd, bridges, containers, and IPv6

I apologize for the quickie here; it’s late, I’m tired, and I just cobbled together a solution to a problem that’s been bothering me since this weekend. First, some backstory since I know you guys appreciate it when I go off on tangents. (And if you don’t, you can just scroll passed this nonsense.)

I’ve been migrating most of my services on my home file server into their own containers. I already did this on my VPS, and I plan on doing it to others eventually, because the service isolation is somewhat helpful and prevents stupid mistakes from doing equally stupid things to the host. Containers aren’t a panacea, of course, and the man pages specifically warn against using them as a security apparatus. It’s not effective, so they say, but I’m also a believer in defense-in-depth.

Anyway, the intent has been mostly a matter of isolation. I have a Minecraft server running, among other things, and I’ve been wanting to isolate it from the rest of the system. Not only does this make configuration somewhat easier since I can have service-specific user accounts without polluting the host passwd entries, but it means I can provide some package isolation as well (no Java on the host–yay!). This isn’t without its shortcomings, mind you, and it’s been one Hell of an interesting battle, particularly when IPv6 is thrown into the mix. But I digress, and the server-specific configuration is a welcome post for another day.

In the coming weeks, I’ll see about making a post on using systemd-nspawn for container isolation, why I chose it over Docker and LXC, how to get it working with user namespaces, and how to configure it with IPv6 addresses routed to each container (rather than through auto-config). Hint: you need routable prefixes like a /48 or /56–trying to (ab)use the neighbor discovery proxy and the respective sysctl settings to segment parts of your /64 non-routable prefix won’t work. Don’t even bother, because it’s not worth the misery.

Meanwhile, back to business: Since I do quite a bit of development of various sorts on my desktop, I also wanted to muck about with containers (which I’ve done for isolating builds), but I wanted to bridge the container with my network-facing interface in a manner that it would happily pull down IPv4 and IPv6 addresses from DHCP. My network, on both IP protocols, is serviced by DHCPv4 and DHCPv6 with more-or-less static assignments tossed out to the appropriate systems with a few exceptions. Specifically, connected devices either get their automatically configured IPv6 address, a static address plus an automatically configured address, or some mix (or absence) thereof. Ideally, containers should receive these addresses as well and behave just as an ordinary OS instance or virtual machine would.

Unfortunately, if you follow the systemd-networkd guides for Arch, you’ll quickly wind up in a circumstance where your network is only going to be half-configured. You might be able to get the IPv4 addresses you expect, if you configure the bridge for DHCP, but your IPv6 range isn’t necessarily going to work even if it does accumulate a set of addresses. Bummer.

Oh, and pay attention to casing: If you enter any of the values entirely in lowercase even by accident, nothing will work. I made this mistake, and my eyes never saw anything wrong because I was too focused on reading “bridge” to notice that it should have been entered as “Bridge.” The obsession, in this case, with the word “bridge” apparently served as an override for sensible parsing that included letter case, but this is a fairly common problem when you’ve been steeped in a similar class of issues for days on multiple systems; it’s a similar phenomenon to semantic satiation wherein words begin to look “strange” if you stare at them too long.

For the purposes of this discussion, we’ll assume that we’ve created files in a similar spirit to the Arch guide for systemd-networkd bridges: vbr0.netdev, vbr0.network, and whatever the name is of your hardware interface (mine is enp6s0, so I named its configuration rather creatively: enp6s0.network).

As with the guide, our files are as follows:

# vbr0.netdev
[NetDev]
Name=vbr0
Kind=bridge
# vbr0.network
[Match]
Name=vbr0
 
[Network]
DHCP=yes

(Notice DHCP=yes above. If you need static assignments, change this accordingly.)

And finally:

# enp6s0.network
[Match]
Name=enp6s0
 
[Network]
IPv6AcceptRouterAdvertisements=no
Bridge=vbr0

Oh! What’s this IPv6AcceptRouterAdvertisements=no? More importantly why do we need it? We want to accept IPv6 RAs on our network, don’t we?

The answer might surprise you: Yes and no. Yes, we need router advertisements, but no, we don’t need them on the bridge’s slave device. (Bonus: You won’t find this in the current documentation.) If you fail to add this option, your physical ethernet device will collect an autoconfiguration address, probably accumulate the appropriate route information, configure itself for IPv6, all while your bridge device does the same thing. You’ll be left with an interface that works for IPv4, is configured for both IPv4 and IPv6, but refuses to do anything with its IPv6 assignments. Obviously, if your network isn’t running radvd or similar, you won’t need this, and you certainly won’t need this if you’re not using IPv6. However, if you’re not running IPv6, you probably aren’t reading this article, are you?

If you’ve made it this far, I assume you’re interested in the final moment of truth and the culmination of our journey together. I therefore present to you a systemd-nspawn container with DHCP assignments for IPv4 and IPv6 auto configuration (addresses redacted for privacy):

[root@buildbot network]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: host0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default qlen 1000
    link/ether 1e:21:2b:fc:51:b9 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 192.168.5.24/23 brd 192.168.5.255 scope global dynamic host0
       valid_lft 938sec preferred_lft 938sec
    inet6 [prefix redacted]:664e:9f3d/128 scope global noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 [prefix redacted]:fefc:51b9/64 scope global mngtmpaddr noprefixroute 
       valid_lft forever preferred_lft forever
    inet6 fe80::1c21:2bff:fefc:51b9/64 scope link 
       valid_lft forever preferred_lft forever

If you haven’t bothered planning your IPv6 migration, you really ought to. The inevitable is coming sooner or later. You do want to be prepared, don’t you?

No comments.
***

Shell Voodoo, Connected IPs, and Counting Total Connections

I’m posting this mostly as a note to myself, but if you, future visitor, stumble upon this post and have improvements or other things you’d like to share, be my guest. Posts that are overly critical of the methodologies provided by others, or those which otherwise add nothing to the discussion will be removed. This is especially true for those espousing beliefs that PowerShell is superior.

I won’t go into the exact details of why we needed to do this, but the general break down is thus:

  • Get a list of connected IP addresses
  • Sort them
  • Count how many connections were made from a single address

Fortunately, the solution turns out to be quite easy. For FreeBSD:

netstat -anfinet | grep -v 127.0.0.1 | awk '{ print $5 }' | \
grep -E '.*([0-9]{1,4}\.)+.*' | sed 's/\(.*\)\..*/\1/' | \
sort -g -k 1 | uniq -c | sort -n -k 1

And for most derivatives of Linux:

netstat -anW --tcp --udp | grep -v 127.0.0.1 | awk '{ print $5 }' | \
grep --color=never -E '.*[0-9]{1,4}(\.|\:).*' | sed 's/\(.*\)\:.*/\1/' | \
sort -g -k 1 | uniq -c | sort -n -k 1

You may need to modprobe sctp to get the --tcp and --udp netstat flags working. Also, both of these should work with IPv6 addresses, too, which is why I’ve tried to keep the sed regex as simple as possible.

What the Eff is This?!

Okay, I agree. I’ve probably made some kind of mistake somewhere; I don’t know awk or sed quite as well as I should (easily fixed, if I ever wanted to spend a weekend learning). That said, here’s my understanding of how this should work. First, we’ll deal with the FreeBSD derivative, line by line:

FreeBSD

Here is a breakdown for the FreeBSD-specific stuff:

netstat -anfinet | grep -v 127.0.0.1 | awk '{ print $5 }' | \

As with all platforms I’m aware, -an shows all connections by their numerical addresses. netstat prefers to perform a reverse lookup on every address, and this can take some time. However, the FreeBSD-specific option -f inet specifies to only show INET (IPv4/IPv6) addresses and eliminates much of the cruft associated with local Unix domain sockets. Likewise, we trim localhost from the list with grep -v, and we fetch the 5th output column using awk

grep -E '.*([0-9]{1,4}\.)+.*' | sed 's/\(.*\)\..*/\1/' | \

Moving on to the next line, we fetch only those lines that contain something that vaguely resembles an IP address with grep -E (I prefer to use -E here since it gives us the extended regex syntax), and we pass the results into sed to strip off the trailing remote host’s port number. Alternatively, you could use something like 's/^\([0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\.[0-9]\{1,3\}\).*/\1/' instead to filter out IPv4 addresses, but since we already know roughly what to expect from the input we can simplify our regex. Furthermore, we also know that the IP address of the remote host in FreeBSD will always have a dot followed by the port number appended, and we can naively remove this.

sort -g -k 1 | uniq -c | sort -n -k 1

Lastly, we sort (generically, with -gunique addresses in our list including their totals, and we sort numerically by the first column (now containing the count).

Linux

Here is a breakdown for the Linux-specific stuff:

netstat -anW --tcp --udp | grep -v 127.0.0.1 | awk '{ print $5 }' | \

Following in the footsteps of FreeBSD, we use -an to display all connected numeric addresses so we don’t waste time running reverse lookups. However, in most Linux distributions, lengthy columns–and especially IPv6 addresses–will be truncated by netstat’s output. To counter this, we use -W to show the wide listing, and we use --tcp and --udp to filter out only those protocols. You may need to modprobe sctp in order to get this to work; if you can’t, this string of commands might still work. Lastly, we filter connections to localhost with grep -v, and we fetch the 5th column using awk Easy enough, right?

grep --color=never -E '.*[0-9]{1,4}(\.|\:).*' | sed 's/\(.*\)\:.*/\1/' | \

In this next line, we use the extended regex feature of grep -E to filter out lines that look somewhat address-y, and we separate the remote host’s address from its port using sed. In this case, Linux appends port numbers using a colon (:), so we have to deviate slightly from the FreeBSD example. Also, since some distros might alias grep with grep --color=auto|always, we use --color=never to eliminate feeding ANSI control characters to sed.

sort -g -k 1 | uniq -c | sort -n -k 1

Lastly, we sort by the IP address using a generic sort (-g), filter out only those addresses that are unique, count them, and then sort by the count column which is now tacked onto the front.

Now we can get a fancy list of IP addresses, how many connections from them are being made to us, and sort them accordingly! Manipulating grep accordingly can re-introduce localhost or remove specific addresses that might not be of interest.

No comments.
***

Quickie: How to Reinstall XP but keep Partitions as Is

I was asked by a friend of mine how to best go about reinstalling Windows while keeping his other partitions intact. It’s really easy, but there are some points where it’s necessary to exercise caution! This little quickie is a guide on how to do exactly that. Read more…

No comments.
***
Page 1 of 3123