So a week or two ago, I was looking around for a decent autosuggest plugin that had behavior similar to WordPress’ tag manager. The best one I found required jQuery UI which, for various reasons, I didn’t want to use. Furthermore, I needed a plugin that had some extensibility and could differentiate between delimited items.

Failing in my searches, I decided to write my own. You can download it from here. You’ll also need the supporting CSS, but it should be fairly easy to draft your own once you see mine. Your backend script will need to supply a JSON-encoded response roughly as follows (PHP-based example):

echo json_encode(array(array('text' => 'Match 1', 'id' => '1'), array('text' => 'Match 2', 'id' => 2), array('text' => 'Match 3', 'id' => 3)));

I’ll be posting a more feature complete back-end example shortly.

XSuggest usage is also fairly basic. Simply bind it to a text input field and pass in some useful options. Here’s an example:

HTML:

<style type="text/css">
.search {
    background: #eaeaea;
    border: 1px solid #ddd;
    border-radius: 5px;
    padding: 10px 20px;
}
</style>
<div class="search">
  <input type="text" name="search" />
</div>

JavaScript:

(function($){
    $(document).ready(function(){
 
        $("input[name=search]").suggest({
            searchUrl: "search.php"
        });
 
    });
})(jQuery);

searchUrl is the only required option that you must pass to the back end. There’s quite a few other options that can control the behavior of XSuggest, although it isn’t complete, and I expect to add a few more–particularly event handlers. The options you can currently set are:

• searchUrl – URL of the back end script that supplies a JSON-encoded response listing matching data for XSuggest.
• searchVariable – Search variable to attach the user’s input to. This is sent via GET (query string) normally, unless you override it; the default is s
• searchData – Optional data to send along with the search. You may use this to tweak the back end’s response.
• searchMethod – Request method. Default: GET.
• cssClass – CSS class of the dropdown field. Default: xsuggest-dropdown. You may change this to anything you want, but be sure to update the CSS you’re using. You have to use this class–that’s not negotiable. It’s used for selectors as well as styling.
• activate – Function called whenever the user activates the control by pressing enter. This is useful for overriding the form’s default behavior and adding the matched text to the page, similar to what WordPress does with tags. The function can optionally accept one argument containing the jQuery object representing the source control (i.e. your input field). This is useful for reading the field’s contents. One example might be: activate: function(element){$(".search").append("
  "+element.val()+"

  ");.

• append – Allow the control to append data from the user’s input. Mostly useful for tags.
• appendChar – Character to use for appending user input to the control. Default: “,”.

I’ll have more complete documentation and a better tutorial posted later. This plugin does have some potential performance issues (see source) and a few other bugs. For now, it’s pretty good for basic usage, but it might do weird things on high latency connections.

Enjoy.

No comments.

» I hate walls of text. Take me to the downloads! «

Updated October 11th, 2010: Added support for Twisted applications and epoll (and possibly kqueue).

Updated December 3rd, 2010: Changed a few things with the patch distribution. See comments for details. Be aware that this information is or will soon be deprecated. Twisted 10.2 is now available along with a number of improvements, and this patch will likely be ported to plugin status. I am leaving this post mostly intact for historical purposes, although corrections have been made to switch the namespace to tx from twistedpatch to clarify further that this patch has absolutely nothing to do with the wonderful folks who write Twisted.

Introduction

IPv6 support doesn’t exist in the base distribution of Twisted Python (the current version as of this writing is v10.1). Over the years, there have been several inquiries related to IPv6 support including a proposed patch, but support for the protocol is still forthcoming. Personally, I don’t like patching a base distribution. After all, it’ll be overwritten the next time I update, and I really don’t want to go through the effort of patching a second time. And who’s to whether or not the patch will apply cleanly in a few months? If you’re not certain this is an important subject, consider that some statistics estimate IPv4 exhaustion will occur in about 230+ days. Food for thought.

My solution is a little different than just simply modifying a handful of files somewhere in $PYTHONPATH and borrows from some of the methods Zope has used for quite some time with their hot fixes. Instead of patching Twisted directly, I have elected to monkey patch Twisted at runtime. This holds several advantages:

• Whenever IPv6 support is finally added to Twisted, it should be fairly simple to remove this patch. Simply change two lines: Replace the appropriate import statement for the Twisted reactor and replace the listenTCP6 method call with whatever Twisted eventually settles on.
• The patch consists of logically separate Python code and is therefore easier to maintain, update, and make other interesting changes to.
• Zope uses monkey patching to issue hot fixes; if it’s good enough for them, it’s good enough for us.
• There’s no need to modify your base Twisted distribution. That’s what this patch does at runtime. This means less hassle every time your distribution pushes an update to Twisted.
• Less code, less duplication. All IPv6 implementations in this patch make use of existing Twisted classes. Nearly everything is provided by subclassing IPv4 Twisted classes and overriding the appropriate methods. This means that IPv6 support should be functionally equivalent to IPv4–when it’s finished.

A word of warning: This will not currently work with .tac Twisted apps and you it has not been tested with a reactor other than the select reactor. This patch only works if you are launching the reactor yourself. If someone wants to take this and modify it to work with twistd, feel free. It shouldn’t be too hard. Also, be aware that IPv6 support appears to be going forward in the Twisted base distribution, so this patch is probably superfluous. I have posted a version of the Twisted patch that should work with twistd. This means you can now use the patch in your .tac Twisted application code! The same caveats apply, of course, and there are some additional usage instructions. See below. You should only consider this as an interim solution. This patch is not a perfect solution, it isn’t the best solution, but it is a solution that precludes you from having to patch Twisted directly.

It’s worth mentioning that this patch uses listenTCP6 instead of outright replacing listenTCP; the latter appears to be the preferred solution. I don’t necessarily agree. Instead, I recommend using the listenTCP6 nomenclature, because it provides the opportunity to listen separately on IPv4 and IPv6 interfaces simply by changing the method call. However, if you wish to use a single method (like listenTCP) to listen on both IPv4 and IPv6 interfaces, simply comment out line 86 in tcp6.py:

            s.setsockopt(socket.IPPROTO_IPV6, socket.IPV6_V6ONLY, 1)

But before you do, you should probably consider the implications of using IPv4-mapped addresses which is what the socket may fall back to using when an IPv4 endpoint is connected. You were warned.

I also argue that adding another method like listenTCP6 is in developers’ best interests because this reason is no longer valid. DNS AAAA records extend IPv6 support to the same hostname an IPv4 address can reference; after all, how does one expect KAME.net implemented the dancing turtle?

Usage Instrutions – Using the Reactor Directly

Usage is exceedingly basic. Extract the twistedpatch into your source directory and replace this import:

from twisted.internet import reactor

With this import:

from tx.ipv6.internet import reactor

This does not import a separate reactor. Instead, what you receive is the exact same reactor import as you would from importing twisted.internet (in fact, that’s what the twistedpatch does) with the listenTCP6 method patched into it. You may then change your application code to read as follows:

if __name__ == "__main__":
    app = MyApp()
    reactor.listenTCP(8080, app)
    reactor.listenTCP6(8080, app) # Add IPv6 support.
    reactor.run()

Usage Instrutions – Making a Twisted Application

As with the listenTCP call, I have committed further blasphemy and added a new class to the Twisted application framework. This allows you to isolate and control IPv4 and IPv6 instances independently without having to worry about IPv4-mapped addresses and other interesting side effects. Remember: This solution isn’t optimal; the Twisted developers would much rather have a single TCP instance to keep the API clean. (I disagree, because IPv6 is a different protocol entirely compared to IPv4… but no matter.) Here’s how you would construct a Twisted application with this patch:

# Import the Twisted service. DO NOT import the internet features.
from twisted.application import service
 
# Import the internet features separately from the twisted patch.
from tx.ipv6.application import internet
 
application = service.Application("myapp")
internet.TCPServer(8000, Application()).setServiceParent(application)
internet.TCP6Server(8000, Application()).setServiceParent(application) # Note the use of TCP6 here

This will launch two separate factory application instances, both on port 8000, using IPv4 and IPv6. You can control each instance independently.

Obligatory Warnings

I have neither presented this patch to Twisted Matrix Laboratories nor have I addressed IPv6 with them in any way. Thus, you should infer that this patch is entirely unauthorized and unsupported by Twisted Matrix Labs. This patch is mostly unfinished and I have yet to run any of Twisted’s unit tests on it. I suspect they will probably fail. It does, however, appear to work rather well with Cyclone:

2010-10-06 14:54:05-0600 [HTTPConnection,0,2001:470:d:407:2d62:6624:eac6:1b96] 200 GET / (2001:470:d:407:2d62:6624:eac6:1b96) 3.29ms

Effectively, here’s what you should expect:

• There is no UDP support. Yet. It should be trivial to add. Simply subclass the appropriate items in twisted.internet.udp and copy the results into tx/ipv6/internet/udp6.py.
• Address resolution as supplied by Twisted will probably break. Frankly, anything related to addresses internally in Twisted will probably break when faced with IPv6. This is expected.
• listenTCP6 will listen only on IPv6 addresses. This is by design; if there is a compelling reason to outright replace listenTCP, I may implement that in the future. As it stands right now, I suspect that IPv4-mapped addresses have a potential to cause more issues than allowing them would otherwise solve. I also don’t believe that this encumbers the Twisted API in any way. After all, Twisted does have listenUNIX for domain sockets–why not add something unique for IPv6?
• Importing the reactor from tx.ipv6.internet instead of twisted.internet should not break existing code. If it does, there’s a problem. This patch does not introduce a new reactor, it simply patches the existing one.
• This patch is fully unsupported by Twisted Matrix Laboratories. Do not submit tickets to them if you have this patch installed; instead, revert the two changes (mentioned above), and test your code. If the problem persists, then it is probably a bug in your code–or a bug in Twisted (unlikely). If the problem disappears, then it’s a problem in my code, and you may complain to me about it. Do not complain to Twisted Matrix under any circumstance unless you are absolutely convinced that it is a problem with Twisted.
• I am not a particularly good Python programmer, and this patch was written hastily in about a half hour. My expertise currently lies in the unfortunate realm of PHP and Java with about 3 or 4 other languages (Python included) added somewhere into that mix. I don’t know enough about Twisted internals or Python to know whether or not I am committing a horrible blasphemy with this patch. Thus, if you don’t like this patch, please submit corrections or write your own. I really don’t mind. I’m just one guy who happens to want IPv6 support distributed as widely as possible, and if this patch can help meet or inspire others to meet that goal, I’ll be happy.

Downloads

If I haven’t scared you off with the notion that this patch might just kick your puppy late one night or bring a swarm of locusts barreling through your neighborhood, perhaps you wish to give it a try. Download it here:

twistedpatch-ipv6.tar.gz
MD5(42e3e8047fbbff165f5a598dbeff7129)
SHA1(a297c882ea2267155e948bcc7d7f2a28a869d953)

twistedpatch-ipv6.zip
MD5(14ecb5ad3dfd9780d1baa8204d907865)
SHA1(ea46ea482aed38e06f1a49dbd672beba45a26dba)

As many of you know I’m a strong promoter of open source. I use a lot of F/OSS (Free/Open Source Software) applications and quite a few commercial ones. In many cases, I have found the the F/OSS apps are useful precisely because they don’t get in the way (OpenOffice.org versus Microsoft Office is a great example where the commercial application is a greater nuisance).

However, I have my share of gripes. I plan on writing a lengthy essay on one such complaint of mine, but that’s reserved for the future! This one is a lot more simple. Here:

Meet Rhythmbox. It ships with Ubuntu (and Ubuntu variants like Linux Mint) as the default media player.

It’s a damn nuisance! It’s uselessness rivals that of Windows Media Player. Though, I think that’s an unfair comparison. WMP ships with useful codecs and is fairly capable in its own right.

Aside from the obvious issue of duplication effort–and there are a lot of free media players on the market these days–it’s increasingly obvious to me that some developers focus more on simplicity and less on usability. Here’s a short use case that they evidently didn’t plan for and it’s really simple. I like to listen to specific songs repeatedly. In fact, I’ll listen to exactly the same song over and over and over and over again until the neighbors lose their mind. If I like a song, I am going to put it on repeat and listen to it over and over again.

You can’t do this in Rhythmbox unless you use the search or artist features to limit the song displayed to one single track. I’m sure the developers would probably suggest that this is intentional–my use case has already been considered–and that the best way is to make sure my current play list shows only one song. I can work with that but it’s inconvenient. If I want to change songs, I have to use the search feature again which completely eliminates the point of having a playlist, album list, or artist list. I kind of like to browse through my list of songs before selecting one to place on repeat, and resorting to a search feature to repeat one track alone is a bit stupid.

Oh, and there’s a HUGE gap between the end of one track and the beginning of another. This occurs even if you have one song on your playlist. The last time I had such a huge gap in playback was when MP3s were first becoming popular and I had an early Pentium clocked at a whopping 200 MHz.

I’m glad I have Amarok to compete against nonsense like this.

Anyway, this rant has a reason. Open source developers often say one of two things: “It works exactly as intended and we have no reason to supply additional functionality” or the dismissive “I write this for my own enjoyment and if you don’t like it, tough!” In the former case, I’m glad there’s competition, and this is precisely why duplication of effort isn’t necessarily a bad thing: The more projects that provide the same service, the more potential competitors there are who are each likely to provide functionality a specific subset of people want. In the case of the latter, if a developer writes the software exclusively for himself and not for the hopes that someone else might make use of it (Pidgin developers, I’m lookin’ at you), that developer has no business encouraging their software to be included in a major public distro like Ubuntu. No, really.

I don’t want to get ahead of myself, though. This rant is one that’s been bugging me for a while now. I’ll have to write it up this weekend!

I think I went a little longer than I expected. If you have corrections or suggestions to make, post them and I’ll make good on any mistakes I’ve made. However, you won’t change my mind: Rhythmbox sucks. I suspect that there are only three types of people who use it: The Rhythmbox developers, poor SOBs who don’t know any better, and people who have extremely limited needs. I suspect the latter group could be mixed in with either of the former.