I’ve been doing commercial software development for vBulletin 3.x (and, by extension, 4.x) off and on now for a couple of years. While there are some things that irritate the crap out of me about both of these products, vBulletin (both versions) have features that just aren’t found in other bulletin board packages. Admittedly, many of these exclusive features are provided by an extensive library of 3rd party software, but the point still stands–as much as I hate to admit it. Few other message boards have a plugin system that’s easy to develop for, and fewer still have the vast library of plugins available. phpBB doesn’t even come close. vBulletin still has its shortcomings for developers, but I’ll save my complaints for a later installment.

What I’m going to write about tonight is something that bit me, and I know it’s going to bite someone else out there: Template hooks have been the bane of my existence in vB4 for the majority of this weekend, and once you start adding a few yourself, you’ll grow to appreciate the manic schizophrenia that is the vBulletin 4.x template system in all its unadulterated glory. I hope to save you from the onset of severe insanity, so keep reading for my story and my solution.

Side note: You might also want to make this a summer project, because you’ll be bald by the time you’re finished, and I understand that bald heads get cold quite quickly. If you’re already bald, accept my apologies and tear something else out–like the upholstery stuffing in your desk chair. Don’t have a chair? Reach for the carpet. Don’t have carpet? Well, you’re on your own.

Template Hooks: They Work–But not When You Want Them To

I’ve written a couple of plugins that rely on the various forumhome_wgo_pos* template hooks for both vBulletin 3.x and 4.x. These hooks work perfectly for most use cases, regardless of when your plugin fires, and are almost foolproof. Don’t be lulled into a false sense of security, though. The moment you do anything unusual with template hooks in vBulletin 4.x, you’ll be bitten by the what-the-heck-happened-to-my-output surprise.

To reproduce the ailment that has been afflicting my sanity for the better part of this last Sunday, I direct you to a simple test:

1. Create a new product, complete with its very own plugin.
2. Set the plugin to fire on the global_start plugin hook.
3. Add the following code to the plugin:

   $template_hook['footer_test_hook'] = '<b>Hi!</b>';

4. Add the following code to your footer template:

   {vb:raw template_hook.footer_test_hook}

5. Run it!

You should notice that you now have a nice, shiny string containing Hi! at the bottom of your page in the footer code. Now, let’s break it:

1. Add a template, such as break_my_footer to your product XML (optional; you could use any other template if you like)
2. Call this template from your plugin using something like:

   $tpl = vB_Template::create('mytemplate');

   Or, if you decided to use an existing (small) template:

   $tpl = vB_Template::create('option');

3. Then modify your template hook code appropriately:

   $template_hook['footer_test_hook'] = htmlentities($tpl->render());

4. Watch in horror as nothing appears in your footer.

Try as I might, I spent a good hour or two trawling various vBulletin support sites for answers. Rather than make a post somewhere and risk having one of their ill-tempered devs explain “Well, this is how it’s supposed to work, didn’t you use the search?” when the built in search generally sucks and Google doesn’t always pick up their help threads, I decided that this issue became personal. That is to say, this code insulted my mother, my father, my nonexistent siblings, and each of my ancestors going back 1,500 years.

After performing various blind tests I concluded that somehow the call to the vB_Template::create() factory method was effectively wiping the contents of $template_hook–or ignoring it, or purging it, or performing an exorcism on it with tremendous glee while I steamed with fury in front of my monitor. I then decided that I’d had enough, and so I searched for the footer template to determine where it was being called, prepared, and possibly rendered in the code. My hunch was that the footer was being generated separately from the forumhome cruft that so happily seemed to work no matter where I used it or what I did with it (and indeed it is generated separately). Yet my own template hook refused to work.

Then I came across this code in includes/class_bootstrap.php:

 $templater = vB_Template::create('footer');
                        $templater->register('admincpdir', $admincpdir);
                        $templater->register('ad_location', $ad_location);
                        $templater->register('cronimage', $cronimage);
                        $templater->register('languagechooserbits', $languagechooserbits);
                        $templater->register('modcpdir', $modcpdir);
                        $templater->register('quickchooserbits', $quickchooserbits);
                        $templater->register('template_hook', $template_hook);
                        $templater->register('facebook_footer', $facebook_footer);
                $footer = $templater->render();

Pay careful attention to the line $template->register('template_hook', $template_hook);. Clearly, the footer is processing the template hook here–so I thought to myself, perhaps there’s a nearby hook that I could attach my plugin to so I can guarantee I know that the content of $template_hook won’t be interfered with.

I scrolled up and found a hook that probably should have been fairly obvious to me from the start. But hey, it’s the weekend. What more can you expect?

($hook = vBulletinHook::fetch_hook('parse_templates')) ? eval($hook) : false;

Sheepishly, I changed my plugin to use the parse_templates plugin hook instead of global_start, and it worked! So the upshot is: If you’re going to try using custom template hooks and you discover that they won’t work the moment you load a template, try changing the plugin hook to parse_templates. It might just fix the problem.

Now, this was admittedly all my fault for not realizing that parse_templates may be the correct solution; I really should have examined the vBulletin sources more closely. Shame on me. In my defense, though, the vBulletin documentation is pretty poor, much of it is outdated, and even less of it focuses on issues specific to 4.x. However, I have one particular bone to pick: It’s puzzling to me that whatever is in $template_hook will work fine up until the moment you decide to call vB_Template::create(). There’s a comment under the create() method that indicates something to do with $template_hook and treating it as a special case for the purpose of various globals or some such, along with a reference to a bug tracker ID. I think that’s more coincidental than anything else, and certainly if I wanted to find out what was happening, I could run a trace with XDebug, but I’m not that desperate–or bored (yet). My guess is that, somehow, subsequent calls to vB_Template::create() clobber the contents of $template_hook by the time vBulletin gets around to rendering the footer; I may be wrong–I probably am–but this is an example of bizarre code suffering from manic schizophrenia.

Frankly, the vBulletin sources are so stupidly convoluted it’s a miracle the software works as well as it does. I’ll save that for another rant much later this week or next. In short, remember: If you’re toying with custom template hooks, you might just break your code. If you do, try changing the plugin you’re writing for template rendering purposes to hook into parse_templates. You’re almost guaranteed to have little to no interference with the contents of $template_hook and the parse_templates hook is nearest to the templates that are most likely to be affected.

Toodles for now. Expect to see a whiny rant soon!

If you’re using a single monitor, this article won’t be of much use to you. For the rest of you who have two (or more) monitors plugged into your box, you may be able to glean something of useful from this post. I’m sure the topic has been written to death elsewhere, and while there’s a few different ways to accomplish the same thing, this is my solution.

An example and the wrong solution, told as a story

First, the problem. If you’re written any GUI applications, I know you’ve done this at least once before: You create your GUI, you start attaching widgets to it, you launch it, you debug it, and about two or three hours into the project, you’re growing increasingly more annoyed with the window positioning. Maybe the window is attaching itself to the upper left corner, or maybe your window manager is genuinely trying hard to do the Right Thing thus leaving your application to appear randomly around the screen in a feeble attempt to cascade against something that doesn’t exist. Frustrated, you do something like this (SWT example):

// This assumes that display has already been created elsewhere.
// app is a class representing the main window for the application.
private void centerOnScreen ()
{
    Rectangle bounds = new Rectangle(0, 0, 0, 0);
    Rectangle desktopBounds = display.getBounds();
 
    bounds.x = (desktopBounds.width / 2) - (app.DEFAULT_WIDTH / 2);
    bounds.y = (desktopBounds.height / 2) - (app.DEFAULT_HEIGHT / 2);
    bounds.width = app.DEFAULT_WIDTH;
    bounds.height = app.DEFAULT_HEIGHT;
 
    shell.setBounds(bounds); // That'll teach 'em.
}

Your application is now centering itself on the screen. Great! No longer do you have to hunt around for the silly thing during debugging to drag it around, resize it, or otherwise mutter unsavory curses under your breath.

You then plug in a second monitor, and immediately those same unsavory words you uttered earlier have grown significantly worse. Now, with a second monitor, your nifty centerOnScreen() method centers the application between windows. “Curses,” you say, “I though I had the damn thing fixed!” Since your boss was kind enough to purchase two identical monitors, you figure you’ll draft up a quick fix:

// This assumes that display has already been created elsewhere.
// app is a class representing the main window for the application.
private void centerOnScreen ()
{
    Rectangle bounds = new Rectangle(0, 0, 0, 0);
    Rectangle desktopBounds = display.getBounds();
 
    bounds.x = (desktopBounds.width / 2) - (app.DEFAULT_WIDTH / 2);
    bounds.y = (desktopBounds.height / 2) - (app.DEFAULT_HEIGHT / 2);
    bounds.width = app.DEFAULT_WIDTH;
    bounds.height = app.DEFAULT_HEIGHT;
 
    // XXX: No one will ever have a monitor greater than 1920 pixels wide.
    if (desktopBounds.width > 1920)
        bounds.x = (desktopBounds.width / 2 / 2) - (app.DEFAULT_WIDTH / 2);
 
    shell.setBounds(bounds); // That'll teach 'em.
}

After such harrowing labor, your application is now back to normal and centering itself on the left-most monitor. You smile smugly, commit the changes, and go home for the day.

The next morning, you get an e-mail from one of the other developers in the office. He’s not particularly happy:

From: Bob Jones <bobj@megacorp.com>
To: UI Design Team <webreakthings@megacorp.com>
Subject: what’s wrong with the ui?

Hey guys, I just noticed that some changes made since yesterday have the application appearing kind of off my left screen. It spills over a bit onto the right monitor.

For what it’s worth, my monitors are of two different sizes so it’s kinda funky.

Oops. Dividing the desktop size in two (you have two monitors of identical dimensions) and then dividing that number in two doesn’t quite work when individual attached screens differ in width. Worse, what happens when someone buys a monitor with a horizontal resolution greater than 1920?

From: Milton Pencilpicker <miltonpen@megacorp.com>
To: UI Design Team <webreakthings@megacorp.com>
Subject: App keeps appearing waaaaaaay off to the side

Guys,

Did you say the app was supposed to start centering after the changes made yesterday? It’s still broken. It centers vertically just fine, but it’s about a quarter of the way over to the left. Just thought I’d let you know.

Yeah, this solution isn’t going to work. You need to account for resolutions on a per screen basis. So what do you do?

Probe some screens

Java exposes individuals screens through the GraphicsEnvironment singleton. The advantage of this method over getBounds() (or the Swing equivalent) is that you can easily determine the default device. Here’s one such example:

private GraphicsDevice defaultDevice;
private int defaultDeviceOffset = 0;
private ArrayList<Dimension> screens;
private int totalWidth = 0;
 
/**
 * Probe attached displays.
 * This method collects data related to all attached displays.
 * For illustrative purposes, we're recording the dimension of each
 * attached screen and recording it in the local screens arraylist.
 * We then add up the total screen width.
 */
private void probeDisplays ()
{
    GraphicsDevice[] devices = GraphicsEnvironment
        .getLocalGraphicsEnvironment()
        .getScreenDevices();
 
    defaultDevice = GraphicsEnvironment
        .getLocalGraphicsEnvironment()
        .getDefaultScreenDevice();
 
    for (int i = 0; i < devices.length; i++) {
        if (devices[i].equals(defaultDevice))
            // Do something when we encounter the default device.
            // One example would be to calculate the total screen
            // width thusfar. For our example purposes, we're going
            // to record the default device offset versus all other
            // attached screens.
            defaultDeviceOffset = i; // Mostly meaningless; sample purposes only.
 
        DisplayMode dm = devices[i].getDisplayMode();
        Dimension d = new Dimension(dm.getWidth(), dm.getHeight());
        screens.add(d);
        totalWidth += dm.getWidth();
    }
}

By recording individual screens, our centerOnScreen method can now be performed on a screen-by-screen basis:

/**
 * Revised centerOnScreen.
 */
private void centerOnScreen ()
{
    // Center the window based upon the default device dimensions.
 
    Rectangle bounds = new Rectangle(0, 0, 0, 0);
    int widthSoFar = 0;
 
    for (int i = 0; i <= defaultDeviceOffset; i++) {
        widthSoFar += screens.get(i).getWidth();
    }
 
    bounds.x = (screens.get(i).getWidth() / 2) - (app.DEFAULT_WIDTH / 2) + widthSoFar;
    bounds.y = (screens.get(i).getHeight() / 2) - (app.DEFAULT_HEIGHT / 2);
    bounds.width = app.DEFAULT_WIDTH;
    bounds.height = app.DEFAULT_HEIGHT;
 
    shell.setBounds(bounds);
}

By collecting metrics from the GraphicsEnvironment singleton, we’ve established what 1) the default device dimensions and and 2) have established code that will automatically correct for centering the application on the default device regardless of whether it is to the left or right of the other monitor or monitors.

Some Improvements

I want to keep this article short, so I’ll offer some suggestions for further improvements.

• If you wanted to restore the window position after it has been closed and restarted, just record the current window boundaries. Assuming monitor positions don’t change between launches, the application will restart from the last position it was closed at.
• It would be trivial to examine the window positions just prior to restoring the previous session to determine if it will be drawn within the boundaries of the desktop. You could either examine each screen individually (more accurate) or by gathering the desktop boundaries regardless of display (less code, less accurate). The best option is certainly to compare the window’s last position with the dimensions and structure of the probed screen devices. You can correct for changes in resolution, missing monitors, and a few other unexpected situations.
• This example only corrects for the most common scenario where desktop monitors are side-by-side. This is the default on Windows without 3rd party software and is generally a safe assumption. For platforms using Xorg (or similar), individual monitors can be to the left, right, top, or bottom of the primary display. NVIDIA’s drives provide fairly fine-grained control over monitor positioning, so if you’re targeting a multi-monitor environment under *nix, you may wish to do further testing and examine window positions based on height.
• This code can easily be adapted to Swing.
• While the sample code won’t work out of the box with JFace, it shouldn’t be too difficult to modify the sample code so it’ll work. Be sure to use getShell() to obtain window locations and dimensions but be mindful that this will probably break if you try to access the shell object during a close action. I’ll be making another post to demonstrate one possible solution.

Should you find bugs or would like to offer corrections to the example code in this article, please feel free to post. Obviously, corrections or suggestions that extend beyond the spirit of the code won’t be included. If you would like to offer criticism, please post full code snippets (even if you’re copying part of what I’ve written in this article) for clarity; should someone new to Java come across my blog, you would be doing them a fantastic service by limiting the amount of vertical scrolling necessary to read the example code and your critiques. :)

I waive all rights to the code in this article and hereby place it into the public domain.

» I hate walls of text. Take me to the downloads! «

Updated October 11th, 2010: Added support for Twisted applications and epoll (and possibly kqueue).

Updated December 3rd, 2010: Changed a few things with the patch distribution. See comments for details. Be aware that this information is or will soon be deprecated. Twisted 10.2 is now available along with a number of improvements, and this patch will likely be ported to plugin status. I am leaving this post mostly intact for historical purposes, although corrections have been made to switch the namespace to tx from twistedpatch to clarify further that this patch has absolutely nothing to do with the wonderful folks who write Twisted.

Introduction

IPv6 support doesn’t exist in the base distribution of Twisted Python (the current version as of this writing is v10.1). Over the years, there have been several inquiries related to IPv6 support including a proposed patch, but support for the protocol is still forthcoming. Personally, I don’t like patching a base distribution. After all, it’ll be overwritten the next time I update, and I really don’t want to go through the effort of patching a second time. And who’s to whether or not the patch will apply cleanly in a few months? If you’re not certain this is an important subject, consider that some statistics estimate IPv4 exhaustion will occur in about 230+ days. Food for thought.

My solution is a little different than just simply modifying a handful of files somewhere in $PYTHONPATH and borrows from some of the methods Zope has used for quite some time with their hot fixes. Instead of patching Twisted directly, I have elected to monkey patch Twisted at runtime. This holds several advantages:

• Whenever IPv6 support is finally added to Twisted, it should be fairly simple to remove this patch. Simply change two lines: Replace the appropriate import statement for the Twisted reactor and replace the listenTCP6 method call with whatever Twisted eventually settles on.
• The patch consists of logically separate Python code and is therefore easier to maintain, update, and make other interesting changes to.
• Zope uses monkey patching to issue hot fixes; if it’s good enough for them, it’s good enough for us.
• There’s no need to modify your base Twisted distribution. That’s what this patch does at runtime. This means less hassle every time your distribution pushes an update to Twisted.
• Less code, less duplication. All IPv6 implementations in this patch make use of existing Twisted classes. Nearly everything is provided by subclassing IPv4 Twisted classes and overriding the appropriate methods. This means that IPv6 support should be functionally equivalent to IPv4–when it’s finished.

A word of warning: This will not currently work with .tac Twisted apps and you it has not been tested with a reactor other than the select reactor. This patch only works if you are launching the reactor yourself. If someone wants to take this and modify it to work with twistd, feel free. It shouldn’t be too hard. Also, be aware that IPv6 support appears to be going forward in the Twisted base distribution, so this patch is probably superfluous. I have posted a version of the Twisted patch that should work with twistd. This means you can now use the patch in your .tac Twisted application code! The same caveats apply, of course, and there are some additional usage instructions. See below. You should only consider this as an interim solution. This patch is not a perfect solution, it isn’t the best solution, but it is a solution that precludes you from having to patch Twisted directly.

It’s worth mentioning that this patch uses listenTCP6 instead of outright replacing listenTCP; the latter appears to be the preferred solution. I don’t necessarily agree. Instead, I recommend using the listenTCP6 nomenclature, because it provides the opportunity to listen separately on IPv4 and IPv6 interfaces simply by changing the method call. However, if you wish to use a single method (like listenTCP) to listen on both IPv4 and IPv6 interfaces, simply comment out line 86 in tcp6.py:

            s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)

But before you do, you should probably consider the implications of using IPv4-mapped addresses which is what the socket may fall back to using when an IPv4 endpoint is connected. You were warned.

I also argue that adding another method like listenTCP6 is in developers’ best interests because this reason is no longer valid. DNS AAAA records extend IPv6 support to the same hostname an IPv4 address can reference; after all, how does one expect KAME.net implemented the dancing turtle?

Usage Instrutions – Using the Reactor Directly

Usage is exceedingly basic. Extract the twistedpatch into your source directory and replace this import:

from twisted.internet import reactor

With this import:

from tx.ipv6.internet import reactor

This does not import a separate reactor. Instead, what you receive is the exact same reactor import as you would from importing twisted.internet (in fact, that’s what the twistedpatch does) with the listenTCP6 method patched into it. You may then change your application code to read as follows:

if __name__ == "__main__":
    app = MyApp()
    reactor.listenTCP(8080, app)
    reactor.listenTCP6(8080, app) # Add IPv6 support.
    reactor.run()

Usage Instrutions – Making a Twisted Application

As with the listenTCP call, I have committed further blasphemy and added a new class to the Twisted application framework. This allows you to isolate and control IPv4 and IPv6 instances independently without having to worry about IPv4-mapped addresses and other interesting side effects. Remember: This solution isn’t optimal; the Twisted developers would much rather have a single TCP instance to keep the API clean. (I disagree, because IPv6 is a different protocol entirely compared to IPv4… but no matter.) Here’s how you would construct a Twisted application with this patch:

# Import the Twisted service. DO NOT import the internet features.
from twisted.application import service
 
# Import the internet features separately from the twisted patch.
from tx.ipv6.application import internet
 
application = service.Application("myapp")
internet.TCPServer(8000, Application()).setServiceParent(application)
internet.TCP6Server(8000, Application()).setServiceParent(application) # Note the use of TCP6 here

This will launch two separate factory application instances, both on port 8000, using IPv4 and IPv6. You can control each instance independently.

Obligatory Warnings

I have neither presented this patch to Twisted Matrix Laboratories nor have I addressed IPv6 with them in any way. Thus, you should infer that this patch is entirely unauthorized and unsupported by Twisted Matrix Labs. This patch is mostly unfinished and I have yet to run any of Twisted’s unit tests on it. I suspect they will probably fail. It does, however, appear to work rather well with Cyclone:

2010-10-06 14:54:05-0600 [HTTPConnection,0,2001:470:d:407:2d62:6624:eac6:1b96] 200 GET / (2001:470:d:407:2d62:6624:eac6:1b96) 3.29ms

Effectively, here’s what you should expect:

• There is no UDP support. Yet. It should be trivial to add. Simply subclass the appropriate items in twisted.internet.udp and copy the results into tx/ipv6/internet/udp6.py.
• Address resolution as supplied by Twisted will probably break. Frankly, anything related to addresses internally in Twisted will probably break when faced with IPv6. This is expected.
• listenTCP6 will listen only on IPv6 addresses. This is by design; if there is a compelling reason to outright replace listenTCP, I may implement that in the future. As it stands right now, I suspect that IPv4-mapped addresses have a potential to cause more issues than allowing them would otherwise solve. I also don’t believe that this encumbers the Twisted API in any way. After all, Twisted does have listenUNIX for domain sockets–why not add something unique for IPv6?
• Importing the reactor from tx.ipv6.internet instead of twisted.internet should not break existing code. If it does, there’s a problem. This patch does not introduce a new reactor, it simply patches the existing one.
• This patch is fully unsupported by Twisted Matrix Laboratories. Do not submit tickets to them if you have this patch installed; instead, revert the two changes (mentioned above), and test your code. If the problem persists, then it is probably a bug in your code–or a bug in Twisted (unlikely). If the problem disappears, then it’s a problem in my code, and you may complain to me about it. Do not complain to Twisted Matrix under any circumstance unless you are absolutely convinced that it is a problem with Twisted.
• I am not a particularly good Python programmer, and this patch was written hastily in about a half hour. My expertise currently lies in the unfortunate realm of PHP and Java with about 3 or 4 other languages (Python included) added somewhere into that mix. I don’t know enough about Twisted internals or Python to know whether or not I am committing a horrible blasphemy with this patch. Thus, if you don’t like this patch, please submit corrections or write your own. I really don’t mind. I’m just one guy who happens to want IPv6 support distributed as widely as possible, and if this patch can help meet or inspire others to meet that goal, I’ll be happy.

Downloads

If I haven’t scared you off with the notion that this patch might just kick your puppy late one night or bring a swarm of locusts barreling through your neighborhood, perhaps you wish to give it a try. Download it here:

twistedpatch-ipv6.tar.gz
MD5(42e3e8047fbbff165f5a598dbeff7129)
SHA1(a297c882ea2267155e948bcc7d7f2a28a869d953)

twistedpatch-ipv6.zip
MD5(14ecb5ad3dfd9780d1baa8204d907865)
SHA1(ea46ea482aed38e06f1a49dbd672beba45a26dba)