Building notes, projects, and occasional rants


last update:

Attaching jobs in Gearman

I've used Gearman on and off in the past but for a new project, I've decided to explore some features I rarely made use of previously. Most notably, the unique ID that clients can submit with each job. Let me just clarify some behaviors for non-background jobs: the worker will execute the job until it finishes even if the client that submitted it dies;if the client dies before the job is passed on to a worker, it will be removed and will never execute.


This all started with an article by Marcel Gruenauer "hanekomu", "Repeatedly installing Task::* distributions". What he wants is a way to tell CPAN this: "install the latest versions of my dependencies". His solution wont work unfortunately. The code that he gives us will prevent the Task:: module from being installed but it will not guarantee that the latest version of the prereqs will be installed in the following runs. the reason is simple: if you don't ask for a specific version of your prereqs, CPAN will accept any version, so it will only install each prereq once, the first time.

AnyEvent::Mojo 0.8

I've uploaded to PAUSE release 0.8 of AnyEvent::Mojo. It should be on your local CPAN mirror in a little while. This was a long time coming unfortunately, and I accumulated FAIL test reports on CPANTS, but its here now. Given that it uses the latest Mojo release, it supports HTTP keep-alive and pipelining, chunked-encoding and 100-Continue requests. Although the test suite passes, I'm not fully confident on the pipelining code. My next step is to write a client with a slow network reader to exercise some corner cases of that part of the code.

bash completion for Github gem

For some time now, I had the Github gem installed (if you want to know more, I suggest a old blog post about the Github gem). This gives you a small gh script that interface to Github APIs and make common operations like creating repositories, cloning, and fetching other repositories in the project network easy and fast. But I'm a lazy bastard, and the lack of a bash completion script was getting on my nerves.

Generating charset_table maps for Sphinx

At work, I wanted to improve the back-office search engine and installed Sphinx. On a regular basis, it indexes all our users, courses, teachers and other important tables. It works great, low install barrier, low maintenance, and it is very very fast. Perfect. One of the problems that we found, that limited the usefulness of the full-text search engine, is that a lot of our text has accents, and it would be better to ignore those.

Tip: use pv to monitor mysql loads

If you are fortunate enough to be able to reload your development databases from time to time with production data, this might help. The usual command you would use is something like this: gunzip -c db-backup.sql.gz | mysql -udevuser -ppass db_dev If your dump is big this can take a while and you wont have a clue about what is happening. Instead, install pipe viewer and do: gunzip -c db-backup.sql.gz | pv | mysql -udevuser -ppass db_dev and you get a nice speed meter.

A new look at Mason

On my way to E5, I have to deal with all the legacy sites that came before it, and the vast majority of them are written in Mason. At the time, we used HTML::Mason 1.05, and only after 1.30-ish (when the internal buffering changes introduced in the 1.10 release where reverted) did we upgraded to something more recent. To get an idea of the size of this Mason project, the current sites have a little over 450 different components across 7 sites (different layouts but same content) and 1 management site.

Bitten by prototypes

I just spent the best part of an hour around a problem caused by the behavior of Perl prototypes. I used the following test case to figure it out: use Test::More tests => 1; use Encode qw( encode decode ); sub u8l1 { return encode('iso-8859-1', @_); } my $ola_u8 = decode('utf8', 'Olá'); my $ola_l1 = encode('iso-8859-1', $ola_u8); is(u8l1($ola_u8), $ola_l1); The output of prove x.t is this: t/x.t .. 1/1 # Failed test at t/x.


Just finished watching the first, and only, season of Firefly). I have only one question: who was the monster stupid that cancelled this show? What was he thinking?

Log::Log4perl tip

I use Log::Log4perl for all my logging needs. Ok, I lie. I use a wrapper that deals with some stuff that I just don't like with Log::Log4perl, but that is a story for another day. One thing that we inherited from log4j was the notion that a message can match multiple loggers in your logging hierarchy. The logic is simple and explained in detail on a Log::Log4perl FAQ entry. If you write something like this in your logger configuration file: