It’s a popular statement in a world where many distributions have standardized on systemd. “I hate systemd” comes the quip–a statement designed to evoke emotion rather than contemplation. Just a mere mention of Lennart Poettering provokes near-Pavlovian salivation in some persons afflicted by such a malady, and they often haven’t the foggiest notion why beyond it’s different.

This post isn’t intended to be a deliberate defense of systemd, although it can certainly be construed as such. There are valid reasons to eschew its use, just as there are equally valid reasons to praise it for its novelty–and its belligerence to upend established convention surrounding sysvinit. What I hope to accomplish herein is to reduce the emotionally-charged nature of the anti-systemd response and convince proponents that opposition to such views isn’t antithetical to strongly held convictions of traditionalism in the context of UNIX design and philosophy. Whether they come away from this in continued opposition to systemd (or not) is largely uninteresting to me; I’d much rather someone walk away with a better understanding of their own views (and opinions) rather than convince them otherwise.

It has been my experience that systemd opponents typically fall into three camps: The first, people who have a limited understanding of systemd but have read volumes of opinion pieces explaining how bad it is, accepting opinions as truth; the second, people who feel it violates traditional mores of the UNIX world; and the third (and smallest) group who disagree with its adoption strictly on technical grounds. The latter group is unlikely to glean anything useful from this post as their opinions are likely founded on reason and experience (though I may disagree). The former two find themselves in opposition mostly by force of ignorance. Fortunately, ignorance is curable (albeit powerful), and worst case, we can lift those readers into the objectivism of the third category: Informed but still in opposition.

Readers who are mostly indifferent to systemd because they’re uninterested in learning it or are satisfied with the sysvinit (or sysvinit-alike, e.g. OpenRC) that was installed with their distribution are not considered the target audience. Though they may collect useful arguments, either for or against, I don’t expect them to find much else of interest. They may opt to skip this post entirely.

systemd Violates UNIX Principles

The concept that systemd grossly violates UNIX principles is an argument that usually establishes a few key points (erroneously): 1) systemd is monolithic; 2) systemd violates the principle of “everything should be text;” and 3) systemd is unnecessary complex. On occasion, this may coincide with a fourth point that establishes systemd as an unnecessary replacement for some subsystems or forcibly requires the use of its own internal replacements (such as for syslog or DHCP).

Of these, the third and fourth (surrogate) arguments bear the most weight. As such, I will address them last.

Is systemd monolithic? Yes and no. Yes, the PID 1 replacement that fundamentally exists as “systemd” in its binary form comprises a lot of moving parts, but it’s helpful to understand that systemd as a service runner exposes a significant number of kernel internals: cgroups, capabilities(7), namespaces, read-only file system views, complex service dependency management, and more. systemd is complex but it’s not necessarily monolithic.

Indeed, browsing through the number of binaries in a typical systemd installation will expose a wide assortment of services that perform one specific task. systemd-networkd, for instance, can manage network interfaces. Its companion, systemd-resolved, handles resolver configuration via resolv.conf (and honestly does a much better job of it than dhcpcd hooks or resolvconf).

Does systemd violate the principle that everything should be text? Not really. Whenever this gripe surfaces, it’s usually framed in the context of the systemd journal which does store its output in a binary format. It can also be configured to forward its data to syslog, but I don’t think this argument matters. journalctl comes with tools that can transparently read its binary form just fine thank you very much, along with filtering options that are arguably more powerful than your typical less/grep inspection can muster. In fact, to head off the argument that it doesn’t use “standard tooling,” I might argue that syslog doesn’t either–you have to use other user space tools to open and search through the logs; tools that have become a de facto standard through longevity. Nay, the difference exists mostly in the reality that systemd-journald’s output can’t be read by a tool the system administrator might author independently. Leastwise, not without some work.

There is a strength in what systemd does that isn’t easily replicated via syslog. As an example, it’s possible to configure remote hosts to pack up their binary logs and ship them to another location for centralized logging. Yes, you can do this with one or more syslog distributions, but it’s not easy. Compare this with systemd-journal-remote(8), systemd-journal-gatewayd(8) and journal-remote.conf(5), and you’ll learn it’s a simplified process that does little more than upload binary blobs to a remote HTTP API. Bonus: Because it’s a binary format log, you can selectively extract entries from a remote journald instance. Yes, really.

Aside: I recognize some astute readers will find it cleverly ironic that I’d reference three separate manual pages in the context of the claim that remote logging is easier in systemd, whereas their favorite solution requires reading one or two (and ancillary works; web searches; or more). The facts aren’t quite so malleable: In systemd it’s a matter of enabling the correct services and applying the appropriate configuration changes. There isn’t much need to do anything else.

Returning to our original train of thought: The third dispute, “systemd is too complex,” is a matter of whether all this complexity is useful. Most arguments against systemd’s complexity tend to focus on everything it can do rather than what it does, and I think this disconnect stems from a misunderstanding of what systemd actually is. In particular, it’s apparent that the complaint isn’t really about the complexity of the entire corpus of what systemd can do. It’s a complaint following from the belief that it does all of this by default. This isn’t true; many of the systemd services (systemd-networkd, systemd-resolved, and the more recently notorious systemd-homed) are entirely opt-in. They aren’t necessary and they aren’t typically activated by default (though some distributions make that choice).

I would argue that this complaint ought to focus instead on systemd’s reliance on dbus for its internal messaging apparatus. dbus itself is fairly complex (it’s a message bus…), but it also allows systemd to do a lot of its work under the hood by using an existing message passing system rather than reinventing its own (a surprise to some!). Perhaps it could be argued that repurposing a desktop bus was something of an ambitious choice on Poettering’s behalf, but again, it’s not an illustration of a violation of UNIX principles. If anything, repurposing existing tools should be praised as an example of avoiding systemic problems common among Not-Invented-Here adherents!

At this point, I would presume this section has implicitly answered, by proxy, the question of whether systemd’s replacement of conventional tools is strictly necessary or desirable. If not, then I would posit that more competition is good. As an example, systemd-networkd is far easier to configure and start than DHCPcd or dhclient. systemd-networkd has supported DUIDs out of the box for quite some time, and if you examine the contents of /run/systemd/netif/leases/*, you can copy the DUID between installations to retain static IPv4 assignments leased via DHCP. I’ve done this. (Yes, you have to chop the identifier up a bit, but that’s beyond the scope of this essay.)

systemd Integrates Itself too Deeply

systemd is an init (PID 1) replacement process. Deep integration is its job.

Okay, I get it: systemd as a whole is replacing “too many” established packages. As an example, it contains replacements for one or more of the following: DHCPcd/dhclient, resolv.conf manipulation tools, NTPD, timezone management (more on this in a minute), syslog (we’ve touched on this), hostname management, cron replacement, and probably a dozen other things that I haven’t thought about while writing this post.

I would argue this isn’t strictly a bad thing. Is competition against current DHCP clients particularly egregious? I’d think not–you can still use them if you like. systemd-resolved takes much of the guesswork out of configuring a DHCP client and its helpers to properly update resolv.conf. NTP clients are in the same ballpark–it’s entirely opt-in. syslogging, well, we’ve touched on that. And so on goes the list.

Of course, cron replacement seems to be a particularly touchy point. I don’t really know why, because many of the DIY distributions like Arch and Gentoo don’t actually ship cron by default. You have to install one yourself. Then you have to configure it. Although I’ve never made much use of systemd timers, I will say that shipping software targeting systemd means that if I also ship a systemd timer, I no longer have to worry about whether someone has a cron correctly setup and configured on their system at all. This means that if they deploy a bare container image (say LXD) containing a recent Debian or Ubuntu image, they can install my software and expect it to perform periodic tasks as required without any further intervention. systemd timers also do a bit more than cron. Suggested reading: systemd.timer(5).

And, well, let’s be honest. The crontab syntax is just a little eccentric. Sure, it has its charm, and it’s very compact and easy to read (ahem–once you learn it), but are we defending cron on merit or by way of inertia? Contemplate that question for a while before you continue.

So: Timezones. What exactly does systemd need to manipulate the system timezone for? Well, the short answer is that it doesn’t. The long answer is that the traditional way to configure a timezone for the system was to either allow the installer to do it or to manually place a symlink at the location /etc/localtime pointing to the appropriate zoneinfo file in /usr/share/zoneinfo. systemd itself doesn’t actually manipulate this file, but it does include a tool (timedatectl) that does this for you. Admittedly, this tool does many other things, but it also configures the local timezone for you. Is it worth replacing manual invocation of ln(1)? Probably not, but it’s not exactly doing anything new or particularly troublesome either.

systemd is Pushing Toward a Monoculture

I won’t deny this is a real risk. As of this writing, 8 of the top 10 Linux distributions according to DistroWatch.com use systemd as their sysvinit replacement.

(Ignoring FreeBSD, because I know someone will say “but there’s 11 on that list!”–even though it’s not Linux.)

This is both good and bad. I’ll start with the bad.

Any time there’s a software monoculture, particularly one that’s controlled by a comparatively small number of people, there is a real danger of locking ourselves into a specific mindset. The irony is not lost on me that this is the reason for one of the chief complaints against systemd: Traditionalists who cling to their archaic script-based sysvinit lock themselves into the mindset that sysvinit should only be done with scripts. systemd, by virtue of its market penetration, presents a quandary that we may eventually conclude systemd is the only way to do things. This is bad, but the fallout from this is comparatively minor versus, say, macOS which has no other system besides launchd (from which systemd drew substantial inspiration).

For one, I would imagine that there will always be distributions using alternatives to systemd, even if only as a token gesture to appease the naysayers. Gentoo, while it supports systemd as an alternative init system, uses OpenRC by default. Slackware has options to use a traditional sysvinit or OpenRC. Devuan formed as a consequence of Debian switching to systemd (although I believe users can choose among other inits). Void Linux, in what is probably one of the most novel approaches of a recent distribution, thumbs its nose at convention and elects to use runit instead. While it’s true that most distributions–arguably consuming the plurality of users–have standardized on systemd, I don’t believe there’s any real risk that systemd will become a true monoculture.

As a developer, I think systemd is a good thing for a couple of reasons. First, I can distribute unit files with my software that will initialize it exactly as I intended, and it’ll work across any system running systemd. Provided there’s a fairly recent kernel in place, I can have access to each of the features (see above) that can be used to harden running processes. I don’t even need to write distribution-specific initscripts to ensure the process(es) start up as I’d expect. Maintainers can be happier too, because they then only need to copy out whatever unit files were included with the distribution, patch them as they see fit (if necessary–usually not), and go about their business. Second, if I target a system with systemd, I don’t have to worry about specialized packages like supervisord. Instead, I can be reasonably assured that the process supervisor for the entire system will do exactly what I want. Process fails? It’ll restart. Complex dependency chain when distributing microservices? No big deal; systemd will manage that for me.

The best part? All of that comes for free.

Does this mean I don’t think we can do better? Of course not. I’ve heard it stated before that, paraphrasing, “systemd is a disaster, but whatever comes after systemd will be better than what we have now.”

There may be some truth to such a statement, but I don’t think the statement itself is necessarily a reflective (or is that reflexive?) truth either. systemd can be improved upon, sure, but it exposes a large feature set that once required special tooling (C wrappers, anyone?). More importantly, systemd can be used now. Not next year. Not in 5 years. Not in a decade. Now.

I think the push-back against systemd is potentially dangerous, because it risks frightening off people who might come into the field with new ideas. Seeing how Poettering has been attacked personally (he’s even received death threats–yes, really), they may decide it’s not worth the trouble. That would be criminal.

If any time a new idea surfaced, we immediately bowed to pressure and confessed the naysayers were right before so much as beginning the journey, we’d still be a tribalistic people wandering the plains.

systemd was Written by Lennart Poettering–Just like PulseAudio

You got me. I have nothing else to say.

No, really. I’ll be honest–I’ve seen this argument before, and I’ve seen it it presented as a straight-faced counter to explain why systemd is so awful. This argument is anything but objective and seeks to paint systemd based on either the personality (or personalities) behind the project and on his previous work. I’m not sure this is a particularly good argument, because PulseAudio does actually resolve some long standing issues with Linux audio.

If you don’t know why, then it’s plausible you’ve never dug too deeply into PulseAudio before. But, to humor you: If I have multiple audio cards outputting to multiple devices, I can easily switch between them (think speakers and headsets) by changing the sink output from the mixer. That’s something you can’t easily do from Windows without opening the sound options and fiddling around with default output devices. In Pulse, it’s literally three clicks: Open the mixer, right-click the application, and select the output device.

Let’s be honest: “I hate PulseAudio” is absolutely not a valid argument against systemd. It’s intellectually lazy. It’s strawmanning.

Conclusion

systemd’s criticisms are certainly not without their merits, and I think its worth looking at its deficiencies in the context of what it does right as well as what it does wrong. systemd isn’t perfect–no software is–but I think there’s an argument to be made that sysvinit and sysvinit-compatible init systems are long in the tooth. It’s good to see that there are distributions exploring alternatives (again, Void Linux) and that many others have standardized on an init system that solves long standing issues with process supervision and dependency resolution.

Once upon a time, I used to rely on supervisord to manage multiple processes and to provide some guarantees that if an application failed, it would be restarted. Before that, I relied on DJB’s daemontools (hello qmail!). Each of these solved the deficiencies that existed in traditional sysvinits–and did a darn good job of it. Having said that, I think it’s time that PID 1 finally take control over process life cycle management. Windows had this for a long time. It’s time Linux did too.