Mark Ramm, one of TurboGears’s core developers announced on TurboGears’s mailing list that they will merge with Pylons! To be more precise, the API of TurboGears will be implemented on top of Pylons. It seems they already made some test/proof of concept that are, as they say “a huge success”. That’s really good news for Python web frameworks development, and a good news for me, as I’ll not have to choose between the two :-

As a sidenote, it seems another framework, CleverHarold has disappeared without anybody noticing. Its domain is parked, and today its Google Group page went off (the last messages were from people asking if the project was still alive).

Update: Noah Gift wrote a nice article about the merge.

As the title says it, each presentation was filmed, and has just been uploaded for everyone’s pleasure! Here is the two-part video presentation of Twisted, done by Michael SCHERER.

[googlevideo]http://video.google.com/videoplay?docid=8739163173039026567[/googlevideo]
[googlevideo]http://video.google.com/videoplay?docid=1837647658089311574[/googlevideo]

Oh, by the way, please take a minute to vote for your favourite Internet Engine!

I went today to the Journée Python 2007 Conference in France. I managed to see half of the Twisted intro, some lightning talks, and most of the afternoon presentations (thanks to Ido’s mid-day nap).

Most of the talks were introductory type, but they were finely presented (alas most of the audience already knew python). I hope I’ll find the time to prepare some more advanced Twisted presentation for next year’s Conference (there’s one, right?)

Pylons is a cool web framework (one more, besides TurboGears, Django, Zope, etc.). Someone posted on the mailing list a reference to a nice introduction.

I found a nicely written article about the problems with Ruby, written by a Ruby user, and why he found Python to be really good. There’s even a quote about Twisted!

And, there are a bunch of things available to a Python guy that Ruby just can’t compete with that are of particular interest to me. Two that come to mind immediately are Twisted and Stackless Python. The former was used by others at TurnTide for creating a really powerful SMTP testing tool and the latter was used by TurnTide’s competitor IronPort to build one of the industry’s best MTAs.

I didn’t knew that IronPort was done in Python, even in Stackless Python!

Finally, after such painful setups, I really never could find a really suitable configuration that could satisfy me. Here are the different methods I tried to implement TurboGears/Pylons or similar WSGI/Python projects (MoinMoin for example):

• FastCGI: So complex to setup, crashes on its own so often, and leaves running processes in memory so have to kill them each time manually to start again with a clean environment. I have to admit it is easier to configure on Lighttpd.
• SCGI: As complex as FastCGI and not so used in the world, but not bad. Too few options.
• Proxy: Redirecting on a local different port gives nore work, and as soon as you have several other virtual hosts, you’ll have to keep a registry of all your allocated ports. Painful, but easy to setup. Maybe be hard to configure if you’re using Zope, and if you need some remote information (ip address of the user for example), you’re dead!
• Direct Access: Configuring you app to run on a local ip alias on your machine and eventually configure your firewall to DNAT on it. Not that complex to setup, but requires access to you OS confiugration and many apps don’t allow you to only listen on a specific interface (MoinMoin allows it, that’s cool)
• mod_python: Loads Python into memory. Everybody shares the same namespace. Dangerous.

Now here’s a new contender, mod_wsgi written by Graham Dumpleton. That’s right, it will not work for every app. Zope, not being WSGI aware, is out of the way, except for Zope 3.x. But most of Python apps can be modded to be WSGI aware (MoinMoin is an example).

PyPy version 1.0 is now available. That’s really a great news, and a milestone for the Python community (and the rest of the programming world). It’s not yet recommended to use it in production, but we’re not that far from that.

Please read the announcement and go in the different links referenced there to learn more about PyPy and what it will change in your life.

Python is so cool.

I had a hard time understanding the function of each field in an ARP packet. The problem is that the fields change of meaning, depending on the opcode field. The two useful ones are for ARP queries (what is the ethernet address of the ip address I’m giving now) and ARP replies (that ip address is located at this ethernet address).

So to fix this problem once for all, I decided to write a python script that shows the different field values when an ARP packet is captured.

There are several libraries available to the pythonista to manipulate network packets. The most known is certainly pylibpcap which is quite old now, and not really object oriented. It is more an adaptation one-to-one of the C libpcap library, which may be useful for some people.
Another library is pypcap, which is like pylibpcap, but much much more object oriented.
pypcap includes a huge quantity of protocols definitions, so it’s really cool to use, especially because it also includes a network packet capture method. There is no included method to send packets, but there are examples of how to do this in the test files.
Another possibility is scapy, which is an extremely complete program (more a program than a library, even though you can use it as a module). The fact that it’s not that easy to include scapy in my own program, even though there’s now a howto. My program doesn’t need all the bells and whistles given by scapy, so I settled on pypcap.

That’s it. Now, for example, 192.168.4.3 wants to get 192.168.4.254’s ethernet address (192.168.4.254 has 00:90:4c:49:00:2a address and 192.168.4.3 has 00:50:70:b4:19:0c), here is the output:

Henrik Thostrup Jensen announced on the Twisted Users mailing list he made two extensions for the Twisted AMP protocol.

Currently there are two types: A dictionary and a list. The types of the element must be specified (key and value can be different in the dictionary), otherwise they are free form, i.e., the keys in the dictionary can have any name, and the list can be of any size. The types can be nested, e.g., you can create a list of list of strings. I use (or will) the latter to return a query result, for which I do not know the row size. This is currently impossible (AFAICT), in the otherwise excellent AMP protocol.

You can get them on http://www.cs.aau.dk/~htj/code/amptypes-0.1.tar.gz

For people that live under a rock, AMP is a new communication protocol for Twisted (added in Twisted 2.5) much lighter/simpler than PB. It is just a request/response protocol over a persistent connection.

For users that don’t yet have setuptools installed (or a too old version) or who want to include a copy of ez_setup.py in their package distribution, here is the one-true-way. This method is already documented on Phillip J. Eby’s setuptools pages, but it’s buried in the middle, and I keep forgetting how to do it. So here it is:

First, go to the root (trunk) of your project, and edit the properties of your SVN folder using the following command:

svn propedit svn:externals .

This will open your favorite text editor where you’ll put the following line:

ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup

Then do a svn ci, and then svn update, you’re all set!
Note: If you use the find_packages method in your project, you’ll have to explicitely exclude the ez_setup folder. You can edit your setup.py file to add:

setup(
…
packages = find_packages(exclude=['ez_setup']),
)