I found a nicely written article about the problems with Ruby, written by a Ruby user, and why he found Python to be really good. There’s even a quote about Twisted!

And, there are a bunch of things available to a Python guy that Ruby just can’t compete with that are of particular interest to me. Two that come to mind immediately are Twisted and Stackless Python. The former was used by others at TurnTide for creating a really powerful SMTP testing tool and the latter was used by TurnTide’s competitor IronPort to build one of the industry’s best MTAs.

I didn’t knew that IronPort was done in Python, even in Stackless Python!

Finally, after such painful setups, I really never could find a really suitable configuration that could satisfy me. Here are the different methods I tried to implement TurboGears/Pylons or similar WSGI/Python projects (MoinMoin for example):

• FastCGI: So complex to setup, crashes on its own so often, and leaves running processes in memory so have to kill them each time manually to start again with a clean environment. I have to admit it is easier to configure on Lighttpd.
• SCGI: As complex as FastCGI and not so used in the world, but not bad. Too few options.
• Proxy: Redirecting on a local different port gives nore work, and as soon as you have several other virtual hosts, you’ll have to keep a registry of all your allocated ports. Painful, but easy to setup. Maybe be hard to configure if you’re using Zope, and if you need some remote information (ip address of the user for example), you’re dead!
• Direct Access: Configuring you app to run on a local ip alias on your machine and eventually configure your firewall to DNAT on it. Not that complex to setup, but requires access to you OS confiugration and many apps don’t allow you to only listen on a specific interface (MoinMoin allows it, that’s cool)
• mod_python: Loads Python into memory. Everybody shares the same namespace. Dangerous.

Now here’s a new contender, mod_wsgi written by Graham Dumpleton. That’s right, it will not work for every app. Zope, not being WSGI aware, is out of the way, except for Zope 3.x. But most of Python apps can be modded to be WSGI aware (MoinMoin is an example).

PyPy version 1.0 is now available. That’s really a great news, and a milestone for the Python community (and the rest of the programming world). It’s not yet recommended to use it in production, but we’re not that far from that.

Please read the announcement and go in the different links referenced there to learn more about PyPy and what it will change in your life.

Python is so cool.

I had a hard time understanding the function of each field in an ARP packet. The problem is that the fields change of meaning, depending on the opcode field. The two useful ones are for ARP queries (what is the ethernet address of the ip address I’m giving now) and ARP replies (that ip address is located at this ethernet address).

So to fix this problem once for all, I decided to write a python script that shows the different field values when an ARP packet is captured.

There are several libraries available to the pythonista to manipulate network packets. The most known is certainly pylibpcap which is quite old now, and not really object oriented. It is more an adaptation one-to-one of the C libpcap library, which may be useful for some people.
Another library is pypcap, which is like pylibpcap, but much much more object oriented.
pypcap includes a huge quantity of protocols definitions, so it’s really cool to use, especially because it also includes a network packet capture method. There is no included method to send packets, but there are examples of how to do this in the test files.
Another possibility is scapy, which is an extremely complete program (more a program than a library, even though you can use it as a module). The fact that it’s not that easy to include scapy in my own program, even though there’s now a howto. My program doesn’t need all the bells and whistles given by scapy, so I settled on pypcap.

That’s it. Now, for example, 192.168.4.3 wants to get 192.168.4.254’s ethernet address (192.168.4.254 has 00:90:4c:49:00:2a address and 192.168.4.3 has 00:50:70:b4:19:0c), here is the output:

I’ll try to explain some methods and tips on how to recover from a mix in db libraries, or from messages like:

• DBERROR: reading /var/lib/cyrus/db/skipstamp, assuming the worst: No such file or directory
• DBERROR db4: PANIC: fatal region error detected; run recovery
• DBERROR: critical database situation

Read the rest of this entry »

Henrik Thostrup Jensen announced on the Twisted Users mailing list he made two extensions for the Twisted AMP protocol.

Currently there are two types: A dictionary and a list. The types of the element must be specified (key and value can be different in the dictionary), otherwise they are free form, i.e., the keys in the dictionary can have any name, and the list can be of any size. The types can be nested, e.g., you can create a list of list of strings. I use (or will) the latter to return a query result, for which I do not know the row size. This is currently impossible (AFAICT), in the otherwise excellent AMP protocol.

You can get them on http://www.cs.aau.dk/~htj/code/amptypes-0.1.tar.gz

For people that live under a rock, AMP is a new communication protocol for Twisted (added in Twisted 2.5) much lighter/simpler than PB. It is just a request/response protocol over a persistent connection.

For users that don’t yet have setuptools installed (or a too old version) or who want to include a copy of ez_setup.py in their package distribution, here is the one-true-way. This method is already documented on Phillip J. Eby’s setuptools pages, but it’s buried in the middle, and I keep forgetting how to do it. So here it is:

First, go to the root (trunk) of your project, and edit the properties of your SVN folder using the following command:

svn propedit svn:externals .

This will open your favorite text editor where you’ll put the following line:

ez_setup svn://svn.eby-sarna.com/svnroot/ez_setup

Then do a svn ci, and then svn update, you’re all set!
Note: If you use the find_packages method in your project, you’ll have to explicitely exclude the ez_setup folder. You can edit your setup.py file to add:

setup(
…
packages = find_packages(exclude=['ez_setup']),
)

I was looking for a program (command-line) that would allow me to check if a Debian system
was up to date against all the offical security annoucements (DSA). Something like the program glsa-check which is available for Gentoo systems.

I just found two, none of them being really useful:

• forgotten_name: It works, but the bad point is that the inner working is to test the upgrade for ALL packages, which is quite slow…
• tiger: They cheated a little, as they made a “static” file used to compare to a filesystem.

So I decided to make my own. Alas, the people responsible for Debian security
don’t seem to give any easy way to get the DSA in a stable/correct way.

Here are the different possibilities, and why they’re hard/impossible to use:

• There’s a “search engine” that is supposed to allow you to search for CVE entries, but it doesn’t work (has it worked at one time?) and now you only get a message “Debian Search disabled”.
• You can get the “latest” security alerts (DSA) from the Debian security page, even in a RDF format. That would be cool, except:
  • It’s just the 15 or less last alerts
  • The contents are just a title, a link, a two words description and the issued date
• There is the security-announce mailing list. Not really practical.

To correct the problem of the 15 or less entries in the distributed RDF file, I took instead the “year” page which gives exactly the same thing but in HTML. Some regexp, and we get the same result as the RDFs, but with the whole list of DSAs (but still not enough information).

Next, to get the detailed data, that is, the affected packages and the corrected version numbers (the most important things) we need to download the corresponding DSA page. For example, for the DSA 1174, you would get the content of the page http://www.debian.org/security/2006/dsa-1174 .

Here begins the fun. That page doesn’t have a static structure at all! Many inconsistencies are making the parsing of the page unreliable.
For example, let’s just start with the DSA number. For example, for the DSA 1174, you find that on the details page, it’s 1174-1.

Next, you would think that with the use of templates, that page would have some kind of fixed format. Que nenni! The text is not always the same. For example, the text ‘has been fixed in’ isn’t always formatted the same way.

About the affected packages, you have a paragraph named ‘Affected Packages’, which is inconsistent with the really affected packages (never more that one package), which can be found later in the page in ‘Fixed in’.

Redhat is submitting its alerts to OVAL, which uses a really nice format and also gives an interpreter for the language. I saw just one or two messages on the OVAL mailing list about debian :-(

Clever Harold is a new python web framework (another one :-) that uses WSGI as its core glue. This allows you to define your own stack of elements (sessions, authentications, compression, templates, etc.).
It is really well done (for a 0.1 version), and, compared to other WSGI based frameworks (like RhubarbTart or Pylons), seems easier to grasp. The big difference (except it’s well organized, thanks to Paste) is that it automatically guesses the required modules, so you don’t need to specify them in your source header.
Take a look at it, and don’t forget to register on the mailing list!