Lior Gradstein's Blog

Synchronization methods of a file with Puppet

Lior Gradstein — Mon, 29 Sep 2008 12:39:32 +0000

Usually, to synchronize a file with remote hosts, using puppet, one would use the following pattern:

file { "/etc/init.d/pvfs2-server":

      owner => root, group => root, 

      mode => 755, 

      source => "puppet:///files/pvfs2-server"    

}

(using “source” to copy data as-is, and using “template(”filename”) to use a template structure as data).

Or, if one wishes to directly set the content of the target file:

file { "/etc/init.d/pvfs2-server":

   owner => root, group => root, 

   mode => 755, 

   source => "puppet:///files/pvfs2-server"    

}

Please note that on the first line, for example ‘file { “/etc/mpd.conf”:’), the “/etc/mpd.conf” is a merge of two functionalities/concepts: Usually, the syntax is clear and simple:

file { mon_fichier_mpd_conf:

     path => "/etc/mpd.conf",

     […]

}

The first line is about the resource description. Its goal is to be able to reference to it a little later from another resources (Notify[], etc.). By putting directly the file path and name (I think it’s identified as a filename and not as a description because there are quotes, or maybe it’s because it begins with a slash?) you mege the “path” attribute with its description.
On the other end, we’ll not be able to reference to it later, if you need to.

You may also need to synchronize a file that is not present in the repository (puppet:///files/*). That usually happens, for example, when that file is regenerated by an external program or an external action (like /etc/passwd).

file {"/etc/passwd":

     owner => root, group => root, mode => 644,

     content => file("/etc/passwd")

}

How to automatically run a script after inserting a USB device on Ubuntu?

Lior Gradstein — Sun, 13 Apr 2008 21:32:01 +0000

Here’s the setup: you have a computer which has a SD port (small memory cards, mostly used in cameras, phones, etc.). You want to get all of your photos from that card automatically as soon as you insert your card (may be a USB key as well), that is, run a script which will search for all of your photos, and copy them to a local directory.

I know, some programs already do this, but they require Xwindow and some ugly Gnome/KDE app. What we’re doing here is a minimal setup, using the least memory.

Normally, on a normal Unix system you would use the included device detection mechanism, which is hotplug. But on Ubuntu (at least Gutsy and Hardy), the de-facto/required way is by using udev, which is an event based system, using rules to fire new events/mount/symlink or run programs according to some patterns defined as “rules”.

So, the first thing to do is to identify your device according to USB events. udevinfo is your friend. So, after looking at the output of dmesg, you’ll see your USB device is available on some sd* device.

So run:

 udevinfo -a -p  /sys/block/sda

(replace sda with you device)

You will get a list of “blocks” representing each layer of drivers. For example:

root@sleek:/etc/udev/rules.d# udevinfo -a -p /sys/block/sda/
looking at device ‘/block/sda’:

KERNEL=="sda"

SUBSYSTEM=="block"

DRIVER==""

ATTR{stat}=="  229567    [...] 456048  3697068"

ATTR{size}=="488397168"

ATTR{removable}=="0"

ATTR{range}=="16"

ATTR{dev}=="8:0"
looking at parent device ‘/devices/pci0000:00/00[...]:

KERNELS=="2:0:0:0"

SUBSYSTEMS=="scsi"

DRIVERS=="sd"

ATTRS{ioerr_cnt}=="0×0"

ATTRS{iodone_cnt}=="0xcc26b"

ATTRS{iorequest_cnt}=="0xcc26c"

ATTRS{iocounterbits}=="32"

ATTRS{timeout}=="30"

ATTRS{state}=="running"

ATTRS{rev}=="3.AA"

ATTRS{model}=="ST3250310AS     "

ATTRS{vendor}=="ATA     "

These are in fact attributes that you can use in your rule to filter the devices. So for example,
to run a shell script when a SCSI device is detected which has a size of 1GB, I’ll use the following line:

KERNEL=="sd?1", ATTRS{size}=="1999872", RUN+="/usr/local/bin/recup_usb.sh"

This line is to be put in a new file in /etc/udev/rules.d, named something like 91-backuptousbdrivetrigger.rules. It’s better to put the file at the end of the rules (the “91″), as not to disturb other module loadings.

The content of the recup_usb.sh file can be:

#!/bin/bash
if [ "${ACTION}" = "add" ]; then

   rm -rf /tmp/x

   mkdir /tmp/x

   mount -t vfat /dev/sdc1 /tmp/x

   find /tmp/x -name "*jpg" -exec /usr/bin/rsync -avz {}  \

              /home/foobar/Pictures/Incoming/`date +%Y-%m-%d`/ \;

   umount /tmp/x

fi

(yes, I know this code is dirty, I should have used mktemp, but you get the point…)
Don’t forget to chmod +x the shell script and reload the configuration with:

udevcontrol reload_rules

For more detailed information about writing udev rules, take a look at the official documentation, or another’s detailed information about doing the same thing.

TurboGears and Pylons will merge! (and CleverHarold RIP)

Lior Gradstein — Wed, 27 Jun 2007 15:20:08 +0000

Mark Ramm, one of TurboGears’s core developers announced on TurboGears’s mailing list that they will merge with Pylons! To be more precise, the API of TurboGears will be implemented on top of Pylons. It seems they already made some test/proof of concept that are, as they say “a huge success”. That’s really good news for Python web frameworks development, and a good news for me, as I’ll not have to choose between the two :-

As a sidenote, it seems another framework, CleverHarold has disappeared without anybody noticing. Its domain is parked, and today its Google Group page went off (the last messages were from people asking if the project was still alive).

Update: Noah Gift wrote a nice article about the merge.

CCT (CalCubeTimer) v0.2 is now available

Lior Gradstein — Tue, 26 Jun 2007 20:29:24 +0000

CalCubeTimer in its newest version is now available! For people who don’t know, CCT is a java program that times for Rubik’s cubing performance. It is really cool not only because it is beautiful, easy to use, has a client mode to connect to a central server (for contests), but because it allows you to plug you stackmat and show its content in full screen!

HOWTO Installation of OpenWRT on a WGT634U usb router Part 2

Lior Gradstein — Mon, 18 Jun 2007 08:05:51 +0000

OK, we now have to configure the network. The goal is to be able to connect to our access point, where we’ll get information to connect to our “sound server” (I call it like that because I’ve still not decided between using mpd+mpc+samba or some more direct access like the use of esd).

We’ll set the wifi interface as a client for a remote access point. We’ll keep the WAN interface with a static ip (here, 192.168.1.1)

Please note, that changing the configuration will make the remote network access as previously used unusable. The only secure access is still through the serial console.

First, edit the file /etc/config/network:

config switch eth0

     option vlan0 "0 1 2 3 5*"

     option vlan1 "4 5"

[…] # localhost config here

[…]

config interface wan

   option ifname "eth0.1"

   option proto static

   option ipaddr 192.168.1.1

   option netmask 255.255.255.0

[…]

config interface lan

   option type bridge

   option ifname "eth0.0"

   option proto dhcp

And then the file /etc/config/wireless:

config wifi-device wifi0

   option type atheros

   option channel 8

   option disabled 0

[…]

config wifi-iface

   option device wifi0

   option network lan

   option mode sta

   option ssid YOUR_SSID

   option encryption none

   option hidden 0

If your access point has some encryption, like WEP, do this instead of the wifi-iface block:

config wifi-iface

   option device wifi0

   option network lan

   option mode sta

   option ssid YOUR_SSID

   option encryption wep

   option key 1234567899

   option hidden 0

You can take a look at the different available options on this documentation. If all is well, you should have an ip address on your br-lan interface (your WHGT634U hub port is bridged with the wifi interface).

Next part will be the most interesting :-) I think I’ll try all the different methods to implement an automatic player daemon. Direct methods, and indirect methods, using a share fielsystem like CIFS.

Reference

Jkx@Home’s article explaining another use of WGT634U

Videos of every presentation of Journée Python 2007 are now online!

admin — Tue, 12 Jun 2007 00:50:16 +0000

As the title says it, each presentation was filmed, and has just been uploaded for everyone’s pleasure! Here is the two-part video presentation of Twisted, done by Michael SCHERER.

[googlevideo]http://video.google.com/videoplay?docid=8739163173039026567[/googlevideo]
[googlevideo]http://video.google.com/videoplay?docid=1837647658089311574[/googlevideo]

Oh, by the way, please take a minute to vote for your favourite Internet Engine!

Security patching Wordpress themes against Cross-Script Attacks (XSS)

Lior Gradstein — Mon, 11 Jun 2007 21:27:02 +0000

A nice article explaining the vulnerability in Worpdress themes (in fact anything using php :-) was published some days ago. Now, there’s even a vulnerability scanner available (done in Perl).

I ran it on several of the blogs I manage, and all of them were vulnerable to XSS (wp-scanner also tests other vulnerabilities) :-(
I followed the advices on blogsecurity’s website and modified all of the search functions I could find. Now wp-scanner doesn’t report any vulnerability (it doesn’t mean it is cracker-proof, but it’s a good start).

So if anybody else uses the Redoable theme like me, you should patch the

header.php

file. Near the top of the file, find the

"Search for"

string, and enclose the

$s

string with the

htmlspecialchars()

method:

Search for

Do the same for the

searchform.php

file:

searchform” action=”

An even better protection would be to use the mod_security module for Apache/Apache2, which can detect and block these kind of attacks. But this requires that you control your server.
To prevent web visitors from sending tags, you can add the following rule in your virtual host:

SecFilter “<(.|\n)+>”

Now, when someone requests < anything >, the visitor gets a 403 error, and in your audit log, you now have:

==36d82a37==============================
Request: www.gradstein.info 82.67.175.56 - - [11/Jun/2007:11:10:56 +0200] “GET /?s=%3Cwpscan%3E HTTP/1.1″ 403 202 “-” “Mozilla/5.0″ - “-”
—————————————-
GET /?s=%3Cwpscan%3E HTTP/1.1
mod_security-message: Access denied with code 403. Pattern match “<(.|\\n)+>” at REQUEST_URI [severity "EMERGENCY"]
mod_security-action: 403
HTTP/1.1 403 Forbidden
Content-Length: 202

Please note, that mod_security does not correct your application. Here if you only use mod_security, Wordpress theme will still be vulnerable on the underlaying level. It is OK to use mod_security, but it is much much more advisable to correct the origin of the problem and not cover it.

HOWTO Installation of OpenWRT on a WGT634U usb router

Lior Gradstein — Sun, 10 Jun 2007 15:00:15 +0000

Before the 7.06 stable version of OpenWRT there were many problems for making USB audio working on this exceptional router (the only one I know with a USB port). Problems with kernel version 2.4/2.6, problems with alsa, problems with USB not working (usb2), etc. Take a look at the forums, and see all horrible, painful modifications/patches you needed to apply to make it working.
I have to say that the OpenWRT developers/contributors (and the forum members) did a wonderful job and corrected every known problems. So with this version, the installation is really straightforward!

Please read the following to get all the details.

I take no responsibility if you damage your router.

This installation procedure has been done with OpenWRT v. 7.06 (stable).

These requirements are not really requirements, but are the tools/material I used to make it work. So with these, you’re sure to make it work, like me :-)

Hardware Requirements

A PC with a serial port
A breadboard
A serial cable that you can unwire
A network cable
A USB sound adapter

Software requirements

A connection to Internet
A Linux OS (I use Gentoo, but any will be OK)
A TFTPd server
A DHCP server
Screen software

Serial updates

First thing to do is build a serial connexion to the WGT634U to be able to flash it. Read the infos about the serial console on this page. I choose to get a normal serial cable, remove one side of the cable and plug it on my breadboard, which is itself linked to the 3 pins on the NetGear.

Install TFTPd, and set it to get its files from

/tftproot

(for example). Put the OpenWRT binary file for WGT634U in it (get it from openWRT’s download section).

Run the following command to catch everything from your Linux ttyS0 port:

screen /dev/ttyS0 115200

Put a network cable on the WAN port of your Netgear, and connect it on your network (which should be linked to Internet):

Here, the yellow network cable is plugged in the WAN port (right).

Unplug and replug the power supply cable, and at the same time press Ctrl-C several times.

You should get a

CFE>

prompt.
Type:

ifconfig eth0 -auto
flash -noheader 192.168.0.44:openwrt-wgt634u-2.6-squashfs.bin flash0.os

This will ask the DHCP server on the network for an ip address to set on the NetGear WAN interface
(you can verify that the link is OK by pinging your TFTPd server:

ping 192.168.0.44

) and then upload the OpenWRT image to the NetGear.
When done, type reboot.

Wait a few minute, for the flash to be reinitialized, then hit enter to get a console.
Set a password for root (which will activate the SSH daemon):

passwd root

.
Edit the file

/etc/firewall.user

and uncomment the two lines that allow SSH (port 22) connections to the WAN interface:

### Open port to WAN
# — This allows port 22 to be answered by (dropbear on) the router
iptables -t nat -A prerouting_wan -p tcp –dport 22 -j ACCEPT
iptables -A input_wan -p tcp –dport 22 -j ACCEPT

Then reboot.
You should now be able to log in remotely without using the serial cable!

Logon and type:

ipkg update
ipkg install kmod-soundcore
ipkg install kmod-usb-core
ipkg install kmod-usb-ohci
ipkg install kmod-usb-audio

Verify that everything is OK by looking into the

/dev

directory. There should be a

/dev/control*

and a

/dev/pcm*

file:

root@OpenWrt:~# ls /dev/pcm* /dev/cont*
/dev/controlC0 /dev/pcmC0D0c /dev/pcmC0D0p

If yes, cool! Otherwise, take a look at the kernel logs (

dmesg

) for any problem.
Now, your USB soundcard should be activated. But that’s not sufficient to make it work. You’ll have to add some symlinks to make alsa work correctly.
First install alsa utilities:

ipkg install alsa-utils
ipkg install alsa-lib

Then create the symlinks:

mkdir /dev/snd
ln -s /dev/controlC0 /dev/snd/controlC0
ln -s /dev/pcmC0D0c /dev/snd/pcmC0D0c

Now run

alsamixer

. You should get an ASCII vertical bar that allows you to set the volume:

That’s all for now. You now have a working WGT63U router which can play sound.

Next part will be about configuring Wifi as a client, and installing mpd (http://www.musicpd.org/ Music Player Daemon) and linking it to a server to allow direct play from a remote server!

References

Search for WGT634U on http://forum.openwrt.org
Nathan True’s howto for doing the same thing

Journées Python Francophone 2007 Conference at La Villette, France

Lior Gradstein — Sun, 03 Jun 2007 20:35:17 +0000

I went today to the Journée Python 2007 Conference in France. I managed to see half of the Twisted intro, some lightning talks, and most of the afternoon presentations (thanks to Ido’s mid-day nap).

Most of the talks were introductory type, but they were finely presented (alas most of the audience already knew python). I hope I’ll find the time to prepare some more advanced Twisted presentation for next year’s Conference (there’s one, right?)

Change in syntax and functionality in Asterisk AstDB functions

Lior Gradstein — Mon, 14 May 2007 10:32:00 +0000

Since Asterisk 1.2+, the methods to access the AstDB database (a berkeley local database) were changed from DBGet,DBPut,DBDel to the merged “simpler” one named Set (except for DbDel which seems to still exist). There was a functionnality that when a key was not found in AstDB, a jump would be done to n+101. So the following would be OK:

exten = s,1,Set(forward_number=${DB(FORWARD/1234)})
exten = s,2,Background(one_local/forward/forward_number_is)
exten = s,3,sayalpha(${forward_number})
exten = s,4,Goto(s-report_mode,1)
exten = s,102,NoOp(Pas de valeur ce qui est normal)
exten = s,103,Goto(s-not_set,1)

But with my newly installed Asterisk 1.4.4, it doesn’t jump anymore. It continues to s,2 :-(.
So I guess I need to add a supplementary line to explicitely test the presence of the key:

exten = s,1,GotoIf(${DB_EXISTS(FORWARD/${F_EXT})}?:102)
exten = s,2,Set(forward_number=${DB_RESULT})
exten = s,3,Background(one_local/forward/forward_number_is)
exten = s,4,sayalpha(${forward_number})
…
exten = s,102,NoOp(Pas de valeur ce qui est normal)