» I hate walls of text. Take me to the downloads! «

Updated October 11th, 2010: Added support for Twisted applications and epoll (and possibly kqueue).

Updated December 3rd, 2010: Changed a few things with the patch distribution. See comments for details. Be aware that this information is or will soon be deprecated. Twisted 10.2 is now available along with a number of improvements, and this patch will likely be ported to plugin status. I am leaving this post mostly intact for historical purposes, although corrections have been made to switch the namespace to tx from twistedpatch to clarify further that this patch has absolutely nothing to do with the wonderful folks who write Twisted.

Introduction

IPv6 support doesn’t exist in the base distribution of Twisted Python (the current version as of this writing is v10.1). Over the years, there have been several inquiries related to IPv6 support including a proposed patch, but support for the protocol is still forthcoming. Personally, I don’t like patching a base distribution. After all, it’ll be overwritten the next time I update, and I really don’t want to go through the effort of patching a second time. And who’s to whether or not the patch will apply cleanly in a few months? If you’re not certain this is an important subject, consider that some statistics estimate IPv4 exhaustion will occur in about 230+ days. Food for thought.

My solution is a little different than just simply modifying a handful of files somewhere in $PYTHONPATH and borrows from some of the methods Zope has used for quite some time with their hot fixes. Instead of patching Twisted directly, I have elected to monkey patch Twisted at runtime. This holds several advantages:

• Whenever IPv6 support is finally added to Twisted, it should be fairly simple to remove this patch. Simply change two lines: Replace the appropriate import statement for the Twisted reactor and replace the listenTCP6 method call with whatever Twisted eventually settles on.
• The patch consists of logically separate Python code and is therefore easier to maintain, update, and make other interesting changes to.
• Zope uses monkey patching to issue hot fixes; if it’s good enough for them, it’s good enough for us.
• There’s no need to modify your base Twisted distribution. That’s what this patch does at runtime. This means less hassle every time your distribution pushes an update to Twisted.
• Less code, less duplication. All IPv6 implementations in this patch make use of existing Twisted classes. Nearly everything is provided by subclassing IPv4 Twisted classes and overriding the appropriate methods. This means that IPv6 support should be functionally equivalent to IPv4–when it’s finished.

A word of warning: This will not currently work with .tac Twisted apps and you it has not been tested with a reactor other than the select reactor. This patch only works if you are launching the reactor yourself. If someone wants to take this and modify it to work with twistd, feel free. It shouldn’t be too hard. Also, be aware that IPv6 support appears to be going forward in the Twisted base distribution, so this patch is probably superfluous. I have posted a version of the Twisted patch that should work with twistd. This means you can now use the patch in your .tac Twisted application code! The same caveats apply, of course, and there are some additional usage instructions. See below. You should only consider this as an interim solution. This patch is not a perfect solution, it isn’t the best solution, but it is a solution that precludes you from having to patch Twisted directly.

It’s worth mentioning that this patch uses listenTCP6 instead of outright replacing listenTCP; the latter appears to be the preferred solution. I don’t necessarily agree. Instead, I recommend using the listenTCP6 nomenclature, because it provides the opportunity to listen separately on IPv4 and IPv6 interfaces simply by changing the method call. However, if you wish to use a single method (like listenTCP) to listen on both IPv4 and IPv6 interfaces, simply comment out line 86 in tcp6.py:

            s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)

But before you do, you should probably consider the implications of using IPv4-mapped addresses which is what the socket may fall back to using when an IPv4 endpoint is connected. You were warned.

I also argue that adding another method like listenTCP6 is in developers’ best interests because this reason is no longer valid. DNS AAAA records extend IPv6 support to the same hostname an IPv4 address can reference; after all, how does one expect KAME.net implemented the dancing turtle?

Usage Instrutions – Using the Reactor Directly

Usage is exceedingly basic. Extract the twistedpatch into your source directory and replace this import:

from twisted.internet import reactor

With this import:

from tx.ipv6.internet import reactor

This does not import a separate reactor. Instead, what you receive is the exact same reactor import as you would from importing twisted.internet (in fact, that’s what the twistedpatch does) with the listenTCP6 method patched into it. You may then change your application code to read as follows:

if __name__ == "__main__":
    app = MyApp()
    reactor.listenTCP(8080, app)
    reactor.listenTCP6(8080, app) # Add IPv6 support.
    reactor.run()

Usage Instrutions – Making a Twisted Application

As with the listenTCP call, I have committed further blasphemy and added a new class to the Twisted application framework. This allows you to isolate and control IPv4 and IPv6 instances independently without having to worry about IPv4-mapped addresses and other interesting side effects. Remember: This solution isn’t optimal; the Twisted developers would much rather have a single TCP instance to keep the API clean. (I disagree, because IPv6 is a different protocol entirely compared to IPv4… but no matter.) Here’s how you would construct a Twisted application with this patch:

# Import the Twisted service. DO NOT import the internet features.
from twisted.application import service
 
# Import the internet features separately from the twisted patch.
from tx.ipv6.application import internet
 
application = service.Application("myapp")
internet.TCPServer(8000, Application()).setServiceParent(application)
internet.TCP6Server(8000, Application()).setServiceParent(application) # Note the use of TCP6 here

This will launch two separate factory application instances, both on port 8000, using IPv4 and IPv6. You can control each instance independently.

Obligatory Warnings

I have neither presented this patch to Twisted Matrix Laboratories nor have I addressed IPv6 with them in any way. Thus, you should infer that this patch is entirely unauthorized and unsupported by Twisted Matrix Labs. This patch is mostly unfinished and I have yet to run any of Twisted’s unit tests on it. I suspect they will probably fail. It does, however, appear to work rather well with Cyclone:

2010-10-06 14:54:05-0600 [HTTPConnection,0,2001:470:d:407:2d62:6624:eac6:1b96] 200 GET / (2001:470:d:407:2d62:6624:eac6:1b96) 3.29ms

Effectively, here’s what you should expect:

• There is no UDP support. Yet. It should be trivial to add. Simply subclass the appropriate items in twisted.internet.udp and copy the results into tx/ipv6/internet/udp6.py.
• Address resolution as supplied by Twisted will probably break. Frankly, anything related to addresses internally in Twisted will probably break when faced with IPv6. This is expected.
• listenTCP6 will listen only on IPv6 addresses. This is by design; if there is a compelling reason to outright replace listenTCP, I may implement that in the future. As it stands right now, I suspect that IPv4-mapped addresses have a potential to cause more issues than allowing them would otherwise solve. I also don’t believe that this encumbers the Twisted API in any way. After all, Twisted does have listenUNIX for domain sockets–why not add something unique for IPv6?
• Importing the reactor from tx.ipv6.internet instead of twisted.internet should not break existing code. If it does, there’s a problem. This patch does not introduce a new reactor, it simply patches the existing one.
• This patch is fully unsupported by Twisted Matrix Laboratories. Do not submit tickets to them if you have this patch installed; instead, revert the two changes (mentioned above), and test your code. If the problem persists, then it is probably a bug in your code–or a bug in Twisted (unlikely). If the problem disappears, then it’s a problem in my code, and you may complain to me about it. Do not complain to Twisted Matrix under any circumstance unless you are absolutely convinced that it is a problem with Twisted.
• I am not a particularly good Python programmer, and this patch was written hastily in about a half hour. My expertise currently lies in the unfortunate realm of PHP and Java with about 3 or 4 other languages (Python included) added somewhere into that mix. I don’t know enough about Twisted internals or Python to know whether or not I am committing a horrible blasphemy with this patch. Thus, if you don’t like this patch, please submit corrections or write your own. I really don’t mind. I’m just one guy who happens to want IPv6 support distributed as widely as possible, and if this patch can help meet or inspire others to meet that goal, I’ll be happy.

Downloads

If I haven’t scared you off with the notion that this patch might just kick your puppy late one night or bring a swarm of locusts barreling through your neighborhood, perhaps you wish to give it a try. Download it here:

twistedpatch-ipv6.tar.gz
MD5(42e3e8047fbbff165f5a598dbeff7129)
SHA1(a297c882ea2267155e948bcc7d7f2a28a869d953)

twistedpatch-ipv6.zip
MD5(14ecb5ad3dfd9780d1baa8204d907865)
SHA1(ea46ea482aed38e06f1a49dbd672beba45a26dba)

Earlier this month, I stumbled across this article on Slashdot and my comments inflamed a few posters with an uncomfortable truth:

Insecurity is your fault.

No, really, it is. Now that the storm has mostly blown over and much of this has been forgotten (plus I’ve gotten some time to sit down and write!), I’d like to explain why.

First, though, a brief recap for those of you who may have missed out on all the excitement.

Memcached: Security Hole or Convenient Misconfiguration?

I won’t explain what Memcached is. If you’re curious, the Wikipedia article details its usage model, behavior, and other interesting tidbits for the curious. However, the important part is that Memcached provides absolutely no method or mechanism by which to authenticate attached clients. Neither does Memcached provide an interface to protect data integrity–that’s your job. The whole Slashdot meltdown occurred when some security researchers discovered that many well-known sites failed to protect their Memcached instances, exposing sensitive data to the outside world including passwords, password hashes, e-mail addresses, and other assorted data typically stored in a Memcached instance. The surprising discovery here isn’t so much that the problem existed but that it afflicted the sites of several well known names including PBS! Really, the system administrators of the sites showcased in this study should be ashamed of themselves. (Maybe; I’m sure many of them were too busy to notice the err of their ways. I’m also sure that a sizable chunk of them felt a little more uncomfortable puckering of anatomical places better left to the imagination than is otherwise healthy.)

I also hate to point out the obvious here, but a simple firewall would have circumvented the issue entirely. The authors of the article in question emphasized this point by appending in their conclusion several strings containing “FW”–and then some. It’s not like setting up a firewall is difficult either, so I’m not sure if this is equal parts ignorance, oversight, laziness, economics, and harsh schedules for the IT staff involved. I don’t think any of these would make for a particularly good excuse. After all, it doesn’t take long to set up ipfw, iptables, and kin. There’s also plenty of documentation and quick-start guides available just a Google search away.

Unfortunately, the Slashdot article was slightly sensationalist and came close to leading readers into believing that running Memcached was a security hole. If you were to take a few minutes (do so now) to browse through the comments, you’ll find that they fall into roughly two categories:

• Security is the responsibility of the system administrator, network administrator, or whomever happens to be in charge of handling the servers, their configuration, and/or network topography.
• Security is the responsibility of the developer to ensure that their application doesn’t ship in a potentially hazardous state such that the exposure of critical data could be made accidentally.

Which statement do you agree with?

Take your time. I’ll be here all day.

It’s Your Fault

If you think it’s the developer’s responsibility to ensure their software doesn’t ship in a potentially hazardous state, you’re wrong, and Memcached is a perfect example why the responsibility of service configuration rests squarely on your shoulders. Of course, there are many reasons why the developers are absolved of responsibility, and I’ll attempt to outline some of them here. Essentially, though, if you’re blaming Memcached for this particular deficit, you’re demonstrating great ignorance of the F/OSS ecosystem.

How dare I point out the obvious!

Package Maintainers Know Everything

Most open source projects, particularly those centered around popular operating system distributions like FreeBSD, Gentoo, or Debian often have specific individuals who, largely on a volunteer basis, maintain a specific software package or packages. In many projects, these maintainers operate rather loosely as a committee whose primary objectives are to: 1) make sure the software installs and runs correctly on their target platform(s) and 2) configure the package to ensure that it suits the specific goals of their distribution (or not, sometimes inaction is as much of a goal as action). This is why package maintainers know everything, and I certainly don’t mean it in a derogatory sense: These individuals, typically picked by the community or by merit, understand both the software they’re maintaining as well as the environment they’re maintaining it for. Package maintainers are an integral part of a particular distribution’s microculture, and they understand what defaults are likely to work best and what won’t. There’s also a certain degree of personal preferences involved but package inclusions and behaviors are almost exclusively influenced by the community as a whole.

When it comes to Memcached, Debian-based distributions tend to configure it to listen exclusively on localhost. FreeBSD and Gentoo both leave Memcached as it ships–not configured and not turned on by default. (Ubuntu enables Memcached as soon as you install it unless you’re running Ubuntu server, in which case it remains disabled.) Since the default–not configured–instructs Memcached to listen on all available interfaces, those distributions that happen to ship Memcached in its default state are therefore the same ones that will expose it to the Big Bad Interwebs the very instant it happens to be enabled. Sort of.

When Configurations… are Not

This is a minor point of contention, but I feel it’s worthwhile to mention because it bears some relevance to the topic of Memcached security. Plus, since several comments on Slashdot made reference to some magical Memcached configuration floating around somewhere in the sky, I confess that I feel this overwhelming desire to point out the obvious. It’s a lot like honking at the motorist in front of you whose blissful ignorance of the surrounding world is responsible for backing up traffic for miles while the unyielding stoplight changes from green to yellow to red and back again–you just can’t help but lay it on thick. I’m also convinced that some of you share this blissful dream-like trance with our hypothetical motorist, because I otherwise cannot fathom why you’d think we software developers are out to get you. (Maybe we are, but that’s another story.)

Okay, here’s the earth-shattering news: Memcached has no configuration file. That’s it. That’s the anticlimactic reality my last paragraph was building up to. I don’t believe I can hear you honking yet, so let me say that again.

Memcached has no configuration file. If it has one, it’s a lie. It’s a lot like that cake. The configuration file is a lie.

Several OS vendors (FreeBSD isn’t one of them–same for Gentoo) happen to ship with their particular Memcached package a file conveniently named “memcached.conf.” To most of us servermonkeys, it would appear that this file has something to do with Memcached–and we’d be right. It’s amazing how that works. Ah, but such assumptions only go so far. You see, “memcached.conf” is just a convenience wrapper. Memcached itself has no integral facilities to read external configuration files and relies instead on command line parameters. In fact, I would argue that vendors who provide a separate configuration file of the sort may be misleading impressionable individuals into believing that Memcached ships with its own configuration file, and that those someones who got caught with their firewall down happened to get the short end of the stick by starting up the service without actually checking that file.

Admittedly, it’s easy to be mislead. Here’s a sample of the Ubuntu (server) Memcached configuration file:

# memcached default config file
# 2003 - Jay Bonci


But, if you were to continue reading…

# This configuration file is read by the start-memcached script provided as
# part of the Debian GNU/Linux distribution.

Oh! Memcached doesn’t read it after all! Good thing I read passed the top two lines, otherwise I would’ve been convinced it shipped with Memcached!

Fortunately, both FreeBSD and Gentoo configure Memcached through their RC system and make it fairly clear that any configuration is done directly to the Memcached daemon. FreeBSD configures Memcached via /etc/rc.conf. Gentoo configures Memcached via /etc/conf.d/memcached. Both of these configurations therefore operate in Memcached’s default mode of listening on all available interfaces.

The question, then, becomes a matter of whether Memcached is insecure by default.

If a Cache Server Listens on all Interfaces in the Forest… is it really There?

Memcached gained its reputation as a light weight, fast, minimalistic memory-based caching server partially by not integrating high-latency features like authentication, data integrity checking, and encryption. If the end user’s environment requires any one of these features, it’s up to the end user to implement them on top of Memcached–and most likely, other solutions. (You can run Memcached over an encrypted connection using a variety of techniques, for example, but they require some effort on your behalf to implement.) Failure to secure a network-accessible service isn’t the fault of the developers of that package. After all, if the server were properly firewalled, it wouldn’t be network-accessible outside the network for which it was intended.

(This, of course, assumes that the attackers don’t have access to your internal network. Granted, if this were the case, I suspect Memcached would be the least of your concerns.)

The obvious point of this entire debate–specifically, the one on Slashdot–is that Memcached shouldn’t be shipped to listen on all available interfaces. Certainly there are great points on either side of this issue, but the fundamental truth is that Memcached’s default behavior isn’t up to us to decide. The developers decide what goes, and I’m sure they have a good reason for attaching to every interface under the sun. In this case, the obvious solution isn’t to argue with me about why it wasn’t your fault you just exposed 15,000 passwords to every 16 year old pimply hacker with an Internet connection. Nay, you should argue with the developers of Memcached and explain to them why their software should listen on localhost by default. Maybe they’ll even agree with you and release a new version that does exactly that. On the other hand, they may just give a hearty laugh in your direction and explain that such a drastic change in default behavior would very likely impact the bazillion or so installations of Memcached that exist out there on the Interwebs the next time some system administrator goes to upgrade.

Generally speaking, it’s better to piss off the people who haven’t done enough research to know how to actually use the software than it is to squirt a healthy dose of Tabasco sauce into the shorts of the very people who probably have several dozen large-scale installations each and enough disposable income to donate generously to your project.

Not surprisingly, that brings me to my next point.

If You can’t Configure that Package Securely… It’s Your Fault

I really hate to harp on this point, but since it seems that there’s so many individuals out there bellyaching over the simple fact that Memcached doesn’t listen exclusively on localhost by default, I really have to hammer it until I drive my point home (or break my hammer).

It’s your fault.

Don’t like it? Tough. Write that sentence 300 times on some college ruled notebook paper and get back to me in the morning. No cheating! I’ll be counting every single one of ’em, too.

The implication of my attitude–or attitude “problem,” depending on how you take this–is that configuring any software you install is your business. If you configure it incorrectly and lose your company a million dollars, it’s your fault. If you configure it incorrectly and drive traffic to your site, earning your company a million dollars, it’s your fault. Conversely, if I write a software package, hypothetically, that advertises itself as some must-have network-connected service that’ll make your servers hum, your teeth whiter, and maybe cure world hunger and you install it without bothering to look at the manual to find out that, holy bat nuggets it really is network-connected, you won’t find a lot of sympathy from me when the entire Internet discovers a few dozen things better left secret just because you couldn’t read a two paragraph instruction manual or secure your server. It’s not my fault that my hypothetical software might be used in an inadequately protected environment anymore than it is the Memcached developers’ fault that their server can be accessed remotely without authentication.

Now that We’ve Established Blame, What Next?

If you happened to read the article(s) the Slashdot summary linked to and the first thing that came to mind was “firewall!”, consider yourself dismissed. You’re free to go home, and you’re getting an A for the course. You needn’t trouble yourself with reading through my inane ramblings, because you probably know a lot more than I do. However, if you’re still fuming out the ears blaming the Memcached developers for defaults that clearly should have been better chosen, you’re staying after class.

I torqued off a number of posters on Slashdot when I uttered these words, so I’ll utter them again: It is your responsibility as a systems administrator to understand the security implications of installing any given software package. If it’s installed on your server–doubly so if you’re the one installing it–you damn well better know what it’s going to do when it starts running. If you don’t have a clue what might happen or how to configure it, then shame on you when someone sniffs critical data from a service you didn’t configure correctly. System administration isn’t intended to be easy. System administration isn’t opening a couple of menu-driven wizards, clicking “next,” and hoping for the best. System administration is about best, secure practices; vigilance; researching and reading; understanding, mitigating, and managing risks; understanding the software you’re using; understanding the operating system you’re using; understanding your network environment and topography; awareness; and always, always, always keeping in mind that anything connected to the Internet is at risk. Sounds like a lot of work, doesn’t it?

“But,” I hear you protest, “system administration is about taking care of the users, resetting their passwords when they forget, and making sure everything is hunky-dory when they pull up their TPS reports! You’re describing a consultant’s job.”

Nope, son, that’s the help desk. They’re the colleged-aged kids up front who daydream of murderous intentions every time someone calls up thinking capslock is cruise control for cool and just can’t seem to get their password right. Consultants are the guys you hire to design (or build) the system. System administrators (probably you) are the guys who are hired to understand the system, fix it when it breaks, and be on call when Godzilla comes gnawing through the company’s upstream. If you think system administration should be easy, there are places you can go to for help.

Why Does it Matter?

Part of the reason I like the minimalistic approach FreeBSD and Gentoo both take to configuring (and I use that term loosely) Memcached is because they make no assumptions–other than the defaults, of course. It doesn’t matter whether you’re running a Memcached instance for a farm of 50 webservers and need to set it up to listen on all interfaces–uh oh! defaults!–since the machine is only attached to a private network. It doesn’t matter whether you’re going to be setting Memcached to use 64 megs (the default!) of RAM or 16 gigs. What does matter is that you’ve looked at the configuration your package maintainer–the guy who knows everything–shipped with the software you installed to figure out what it’s going to do. Better yet, it helps to read the manual page. The world isn’t full of pop-out coloring books, and if you’re going to maintain systems for a living, you’re going to have to do some reading. Manpages are a fantastic place to start.

I’ve read the excuse on Slashdot more often than I care to admit that the reason Memcached should listen by default on localhost is because there are far more instances installed by mom and pop web sites that just expect to install it and have it working than there are large scale installations that listen on multiple network interfaces (well, 0.0.0.0:11211 or :::11211). Is that true? I don’t know. Maybe there’s a significant number of webserver admins out there who have heard about this Memcached business and know only enough to know they need to install it. Maybe there’s a lot of people who just run apt-get install memcached, emerge memcached, or whatever install script happens to float their collective fleet of rubber rafts because they like to watch a bunch of text fly across their screen. I have no idea.

I do know that there’s a huge problem with the assumption that most administrators installing Memcached aren’t aware of the security implications. Why? Simple:

• Memcached is a semi-niche market solution for a large scale problem. By the time you start looking around for a solution to offloading some of your server load elsewhere, you’ve probably done enough research to know what Memcached is and why you need it.
• If you’re administering a large site that needs Memcached (see previous point), you’re probably well aware of network security. You wouldn’t let the world connect to every port on your home computer just for fun, so why would you let them do that to your web server?
• Most system administrators are competent, careful, methodical individuals who consider their server farm something like their own children. If we lesser beings so much as tried to touch their machines with our grubby mitts, they’d chase us out of the data center with a shotgun. This whole problem, and the article linked to by Slashdot, doesn’t mention these guys. Those same guys who spend sleepless nights making sure db-140.cluster.lax.internal-net.megacorp.com is running smoothly are the same guys who firewalled their Memcached servers and were not noticed in the report. They couldn’t be, because the report was about incorrectly configured servers. The world is full of surprises, and it may surprise you to learn that there are competent individuals out there.

The other excuse that drove me to the verge of insanity was the proposal that Memcached is a great object store that can increase the performance of a mom and pop webserver, so it’s just as likely to be installed on bobs-fancy-widgets.com as it is on a massive load-balanced site. Thus, since this target market likely doesn’t know the first thing about installing a webserver, Memcached should be set up safely by default. Fine–maybe this is true–but it also assumes that bobs-fancy-widgets.com isn’t firewalled. It should be, and if it’s not, then I’m afraid there’s not much any of us can do for him. Maybe Bob doesn’t know anything about firewalls. Maybe Bob is also receiving a gazillion SSH scans from China, too. Just wait ’til they find out his password is “password”.

I don’t have the data available that these security researchers do. Therefore, I have to base my assumptions on the sample data they released. The companies affected by this security oversight (it’s not a hole) weren’t small fly-by-night organizations. These were just about everything from trendy start-ups to major television network webservers! My guess? It was probably an honest mistake more often than not. Someone forgot to enable the firewall or configure Memcached. They got burned. It sucks, but it happens often enough to serve as a reminder to the rest of us.

Lesson Learned

There’s a valuable lesson to take away from this, and while I realize this essay is reaching almost epic proportions, it’s important enough to warrant just a few more sentences. Never make assumptions about your software’s configuration. If you’re running a business or being paid by a business to run their machines, you should always double-check (or triple-check) the configurations of software you’ve installed to ensure they’re going to run as you expect them to. Make certain to read the appropriate manpages for your software, too, because they often have valuable insight into what the software does or how to configure it. Search the Internet. Oh, and never forget–if you’re using a *nix-based system, there are tools to help you understand what’s running and what’s not. Use utilities like ps and netstat until you can read the output of these programs by heart. netstat is of particular importance; with it, you can understand at a glance what your system is reporting as listening on network interfaces. (-p helps, too, as does understanding similar non-Linux tools like FreeBSD’s sockstat.)

Remember: If some 16 year old hacker discovers the CEO’s password and sends out “Your Fired!” e-mails to the company, wipes the database server, and turns your web server into an archive that’d make the Pirate Bay blush, you’re the one who’s going to get that uncomfortable call at 3:00AM. The Memcached devs? Well, they’ll be sound asleep while you’re sweating bullets. After all, that’s what you’re paid for, right?

Well, well, well, well, well… would you look at that. It appears that Thyth of Tribes/Tribes 2 hackery fame pulled off a fantastic miracle fittingly called TribesNext.

I’m impressed.

For those of you who don’t know: Tribes and Tribes 2 were amazing multiplayer, online-only team-based FPS shooters. The difference between Tribes and other shooters? A jetpack. It always amazed me to have a third dimension of freedom in a shooter that never existed before–or since.

Unfortunately, Dynamix (the studio responsible for Tribes) was shutdown and disbanded by Vivendi, leaving the franchise to effectively rot. Worse, Tribes 2 was one of the first highly anticipated online-only titles to include fairly draconian DRM prohibiting its use online without a valid account (though LAN games could still be played unauthenticated). The DRM was never stripped out after Dynamix’ breakup and as of 2008, Vivendi took down the authentication servers rendering Tribes 2 impossible to use in online matches. In effect, Tribes 2 is the oldest title (and possibly first) whose publisher doomed to the pages of forgotten history.

Enter an individual I know only as Thyth. I recall seeing a site of his back in mid-2008, and he claimed that he could muck about with the logic in the Tribes 2 binary which has thusfar rendered multiplayer unusable. Not only did he manage to circumvent the master checks, but it’s pretty apparent that he also reverse-engineered the protocol used for the master servers and has the client patch working to communicate with his masters. but he has also implemented his own master protocol via HTTP using a Ruby interpreter shoehorned into the executable.

This guy is pretty amazing.

Anyway, I haven’t tested it yet, but if you want to grab the patch to make Tribes 2 usable again, go here.

Edit May 31st – June 1st

Well, the disappointing reality of TribesNext is that it doesn’t play nicely with Wine. Thusfar, I’m relegated to running Tribes from a VM. Josh and I tried for much of this weekend to get it working, but it appears that something in the latest version of Wine (though we tried 1.1.29, 1.1.30, 1.1.40, and 1.1.44) breaks TN’s hack. It mostly works, but the embedded interpreter either dies or issues a response the client doesn’t understand immediately after connect. I don’t know enough about Tribes or its protocol to understand what’s going on, but it certainly appears that Wine (perhaps its implementation of winsock) acts strangely.

However, it works perfectly under a virtual machine. It’s quite amazing! It’s also quite nostalgic. To think: It’s almost been a decade since Tribes 2 was released. I can’t even believe it’s been that long in retrospect…

Regardless, I have some modifications I’d very much like to make to the default scripts that ship with the game mostly for trial purposes. It should be quite fun–especially since I know a great deal more about coding practices in general, and their scripting language isn’t all that difficult.