Pages: 1 2 3 4

Phase Two

I had the nagging suspicion that some of the Python standard library classes for handling network connections may have been somewhat slow. Thus, I performed a second round of tests using a very simple select()-based HTTP server. I also wanted to test the performance of Paster’s built-in HTTP server to provide a baseline performance level for Pylons since the WSGI-dependent Pylons server suffered from terrible performance. The charts I have posted for this phase of benchmarks include all of the previous data slightly dimmed to enhance visibility for the new series of tests. The results are surprising.

Tests Round One: Concurrency 20, Connections 2000

As with the previous phase, the first serious of tests used a concurrency limit of 20 simultaneous connections for 2000 connection attempts in total. Here’s the graph:

Several interesting artifacts appear in this benchmark that are of interest. First, using an incredibly simple select()-based HTTP server appears to suggest that Python’s network performance is about as good for moderate concurrency as PHP acting as a passthrough host and Tomcat serving behind mod_jk. Paster’s server module clearly has the worst performance of the lot, with Pylons operating behind WSGI and Stackless using stacklesssocket.py trailing closely behind. In all fairness, Paster’s HTTP server is intended exclusively for software development and testing, but it’s curious that Paster’s performance matches fairly closely with Pylons’ performance while operating behind a WSGI-enabled Apache install.

Another interesting difference to note is the slight performance increase from Stackless. The version of Stackless used in these tests is based on Python 2.5.1, the same version of base Python installed on my development machine. Ideally, there shouldn’t be a difference in performance capability of around 200 connections per second. It’s possible that eliminating Python’s dependency on the C stack in Stackless Python might be responsible.

Tests Round Two: No Concurrency, 500 Connections Total

Following through with the sequential connection testing of phase 1, here is a graph of server performance including phase 1 tests as slightly dimmed, dashed lines:

In this series of tests, it is perhaps most interesting to note that the performance degradation impacting other server types for sequential connections did not affect my simple select()-based HTTP server as severely. I believe this is most likely due to the incredibly simple structure of the code–no processing is performed behind the scenes–and additional complexity would likely cause it to suffer similarly to the other services. In this particular test, Python + BaseHTTP outperforms the select() server (standard Python) very slightly but lags behind the Stackless select() server. All three perform approximately as well as PHP, slightly better than Tomcat + mod_jk, but significantly worse than Apache and Tomcat (base Tomcat serving HTTP requests directly). Interestingly, Paster’s HTTP server (again, for development and testing only) serving up a Pylons application performs nearly as well as the same Pylons configuration sitting behind a WSGI-capable Apache install.

Conclusion

Python’s standard library socket server and HTTP server modules appear to work fairly well for sequential connections, but their performance drops dramatically whenever connection concurrency increases. Stackless Python clearly and consistently outperforms standard Python in tests where the running code is identical (no dependencies on Stackless are present). I’m not privy to Python internals, but I would expect this difference in capabilities is inherited from Stackless’ design. The most important point has little to do with Stackless or with standard Python; instead, it would seem that Python is fully capable of serving as a robust and capable network server. Unfortunately, the standard library may include additional cruft that weighs the language down. Frameworks like Pylons are indeed very powerful tools, but with all their layers and layers of indirection, performance penalties seem to be their greatest enemy.

Pages: 1 2 3 4