« April 2008 | Main | June 2008 »

May 31, 2008

Scripts repository

I've started to publish my ~/bin script stash at GitHub.

Not all of them are written by me. For example, I've included Gruber's TitleCase.pl script under a new name.

Before you ask about the strange long names, my memory sucks, so I use tab completion a lot. I name all my script with a x- prefix so that I can just type x-<TAB> and see my entire script repository. I use long and (I hope) self describing names.

Common scripts that I use every day get single line shortcuts using shell aliases.

To use, just clone the repository, and add the bin/ to your PATH.

Patches welcome. Recommended workflow:

  1. sign up at GitHub;
  2. fork my project (small fork icon);
  3. clone your fork to your local workstation;
  4. hack, hack, hack;
  5. git-push
  6. hop onto GitHub and send me a pull§ request.

Snapz Pro X a bit to eager?

Run this:

sudo fs_usage  -w -f filesys 'Snapz Pro X'

Does your copy checks the Preferences and License File every two seconds? Sheeeshh...

(tested on Tiger by the way, not sure if it work on Leopard)

May 30, 2008

Making better use of the XMPP presence in Bots

The current crop of the XMPP-based bots is pretty basic. They provide an online presence in the XMPP network, and you interact with them by a command-line-style interface.

These are some suggestions that bot authors can use to increase your XMPP presence easily. I'm going to stick to easy stuff only, most of it widely deployed.

The first thing that you need to understand is that you can have different per-contact presence information. You can switch the status message and the avatar for each of your contacts. Its called directed presence (basically include the contact JID in the to attribute, but you can always check the spec for directed presence).

You can use this to provide a personalized status message for each person on your roster. If your service has a notion of context, you can use the status bar to show the current context. For example, imagine that you have a ticketing system bot. You type "working on TICKETID", and your bot can change the status message to "Working on: TICKETID - TICKET_TITLE (URL of ticket)".

Optionally you could update the status each five minute with the elapsed time since you started working on it.

You can also switch the avatar. Maybe switch to a red background, meaning that you are busy doing something, or putting it in another way, the bot is telling you that he thinks you are working on something.

Other stuff you should do is to provide an alternative representation of the information you send textually. Twitter (when it used to work...) was a great example of this: every tweet you received by IM, included an Atom entry with the most important information of the tweet. This allows client to improve their interaction with such services, by using a more appealing interface.

But Atom entries are not yet standardized by the XMPP Foundation (there is some effort on Atom over pubsub that could be used as an example though). On the other hand, Data Forms are a standard, and more, they provide interaction possibilities.

You could send a "New ticket" notification, including a data form with the structured information, and a select-box with the possible next steps. The user would see the form, and submit back their decision.

Still in the subject of messages, you should know that there are several types of messages. You have chat, headline and normal (strangely enough the default). There are two more but not important in this context. You should use the correct one depending on the type of message you are sending.

If your system is just sending notification, please use the headline type. It makes it clear to clients that they are not chatting with you, just notifying of something that happened. For example, a smart client can react to those messages differently based on your status: in away, xa (extended away) or dnd, a client can keep a log of new headlines and show it like a email client when you return, without interrupting your workflow, probably using a badge with "You have X pending headlines".

The final recommendation: respect the user status. If the user is dnd, are you sure you should send him anything? Why not change the status message to "You have 5 messages pending. Send 'pending' or go to URL to see them"? Maybe even change the avatar color. You should not disturb someone in dnd. I would also say that away and xa are off limits but that's just me.

On the other hand, you should send "Hey there, welcome back! You have 5 pending messages. Send 'pending' to see them or hop on to 'URL'." when the presence changes to chat or available (not a real <show> value, just lack of <show> value. See the valid values of <show> and their meanings in the spec).

Have fun implementing your bots.

git email workflow

I personally was never on the receiving end of a email-based git workflow so I don't know how hard it is to use git to track all the incoming patches and the status of each one.

I much prefer the workflow based around pull-requests, and the GitHub interface for them is extremely nice. Today, as I stood in the shower (it was a shower day, yes, and my best ideas usually hit me under running water) I started to think about a Web interface to manage email-based workflow's. This is a brain dump. I've tried to make it consistent, but I'm not sure I've succeeded.

I'll use GitHub as a target of this feature set, although I don't know how well it would integrate with their view of the world. The fact is that this system could be implemented with a local tool with a web interface and a POP3 client just for personal management of patch queues.

GitHub would provide one or mode email addresses associated with each project. These email addresses are basically treated as patch queues.

Collaborators would send patches by email (using the git-send-email command) to one of the queues. The system would track each patch individually having a state (new, under discussion, rejected, accepted, integrated, for example), and a discussion thread using the same comment system that commits have today. Each patch would be given a patch ID, basically the URL of the patch in this system.

Authenticated users could also add Signed-off-by: headers to each patch.

Patch authors could resend patch sets or individual patches. They could also include a Replace-Patch: header that would automatically link the two patches in the same discussion.

The queue administrator can also group patches into sets, both with the help of the patches Subject lines, or in a ad-hoc manner.

Each patch and each set has an HTTP URL that would return a mailbox formated to pipe directly into git-am. Only accepted patches should be included in the generated mailbox. Optionally, a Patch-Id: header could be added for future tracking.

Applying the a patch or patch set would be as simple as

wget URL-of-patch-or-patch-set -o temp.mailbox
git-am temp.mailbox

If this takes off, we could even patch git-am to accept URLs...

Alternatively, we could write a porcelain tool that would do:

  • record the HEAD ref;
  • fetch the mailbox URL;
  • call git-am with the mailbox file;
  • scan the log since the recorded ref. For all applied patches, signal GitHub that the patch was integrated.

The system could also use GPG to streamline some workflow's. For example, given a GPG white-list or a "trusted" signature (this one is more complicated because you would need to have the private key at GitHub), we could limit queues only to GPG-signed patches, or auto-accept them on the general submission queue.


Anyway, this is a brain dump. I don't use the mail workflow so I don't know how important this would be. I like this idea a lot.

May 28, 2008

Fluid.app + Google Gears = Sweet!

This is great news: Todd Ditchendorf just released a nightly build (link removed) of Fluid.app that includes Google Gears.

I'm a big fan of Google Gears, and Fluid is getting a lot of love regarding integration with the desktop.

BTW, this is a good reason for me to upgrade to Leopard.

update: Todd giveth, Todd taketh away.

TextMate: a new take on Search with Ack

Apparently Trevor Squires found my Search in Project With ack TextMate command at the time it was 404, so he wrote is own.

I haven't tried it yet, but from his description its a lot better than mine.

May 27, 2008

Dan Geer at Source Boston 2008

Watch the video of Dan Geer at Source Boston 2008. Worth every minute of it.

Will 10.5.3 win me over?

For the first time since 10.1.x, I didn't upgrade to the latest Mac OS X with the .1 release. Usually I wait for a .1 and "nuke and pave" my Mac, but this time, after almost 8 months after the 10.5.0 release, I'm still running 10.4, and reasonably happy with it.

Sure, there is software out there that I would like to try but requires 10.5, only its nothing that I totally depend on, so 10.4 is still perfectly usable for me.

But 10.5.3 should be here soon, and with almost 220 fixes, it might be the stable release that the more conservative of us were waiting for.

As usual, I'll wait a week or two after 10.5.3 to make the jump, but I think I will finally unwrap the Leopard DVD sitting on my desk.

Still waiting...

git usage vs operating system

A small meaningless statistic (but interesting nonetheless) about the split of operating systems and git usage on github.

I admit that the second place was a surprise to me. In a good way.

May 26, 2008

Textmate: Search in Project with ack

Just a quick note to point out that my "Search in Project with ack" command for Textmate was updated.

May 14, 2008

My keyboard shortcuts

Inside the "Keyboard & Mouse" preference pane of Mac OS X, you'll find a tab named "Keyboard Shortcuts". This is one of my first stops after any nuke and pave setup of my Macs.

I don't have many shortcuts:

My Keyboard Shortcuts

The Take Rich Note and Append Rich Note integrate all applications with DevonThink. I select what I want to keep, and hit the proper sequence to save it in my default database.

The Quit Safari shortcut prevents me from closing Safari by accident. Its specially useful if you have a lot of tabs open. Recently Safari gained options, like the Reopen All Windows From Last Session, that make this less useful but I still use this.

The Select Next Tab and Select Previous Tab work around the fact that the default shortcuts in Safari for those options just don't work with some international keyboards (like my own, PT-layout).

The final shortcut is something new, that I'm trying out. It gives you a global shortcut to Zoom any window. Not sure if it's a keeper.

Facebook gone XMPP?

Came across this just now: Using Facebook Chat via Jabber

Thats a lot of new victim^H^H^H^H^H^Husers to the XMPP network.

This, and the fact that my Google Alerts on XMPP is now showing about 10 hits per day (from less than 10 per week just 18 months ago), and I would say that XMPP is turning out to be one hot technology at the moment.

Update: given that the news came up via the official Facebook developers site, some people have IM'ed me asking why the question mark in the subject. Well, I'm waiting to see if they will open federation or not. Using XMPP is more than allowing Facebook users to use their favorite XMPP client to chat with the other Facebook users. Without open federation, this is just another private IM network and I think we have enough of those.

So the question mark is there until a answer is found for "Will Facebook enable open-federation over XMPP?"

May 11, 2008

Yada, yada, yada

From the quickies department, the Stubby exception generators (search for stubby) made it to Perl 5.

May 08, 2008

X-Hacker

I was checking something in the Wordpress site:

melo@MrTray:melo $ lwp-request -U -s -S -e  -m HEAD http://www.wordpress.com/
HEAD http://wordpress.com/
User-Agent: lwp-request/2.08

HEAD http://www.wordpress.com/ --> 301 Moved Permanently
HEAD http://wordpress.com/ --> 200 OK
Connection: close
Date: Thu, 08 May 2008 10:48:28 GMT
Server: nginx/0.6.29
Vary: Cookie
Content-Type: text/html; charset=UTF-8
Client-Date: Thu, 08 May 2008 10:48:36 GMT
Client-Peer: 72.232.101.43:80
Client-Response-Num: 1
Client-Transfer-Encoding: chunked
X-Hacker: If you're reading this, you should visit automattic.com/jobs and apply to join the fun, mention this header.
X-Pingback: http://wordpress.com/xmlrpc.php

Nice way to do job hunting.

May 01, 2008

Async DBI

A big part of my work is doing non-blocking servers using the AnyEvent Perl module (I used to prefer POE, but AnyEvent beats POE both in terms of performance and simplicity).

One recurring problem is using DBI inside a non-blocking server. The modules that you can find on CPAN that deal with this problem use slave processes to run the DBI code, and then a lightweight protocol to send the requests back to the master process. This way the DBI can block as much as it wants without causing problems to the non-blocking master process.

This works of course, but I would be lighter to skip all this process mumbo-jumbo if we could have a non-blocking DBI.

I browsed the DBI code and sketched a small API to allow you to execute_nb a statement and receive the results via a callback, but I slammed into problems with the DBD's.

For example, DBD::mysql uses the C-based mysqlclient. But this library is pretty much useless in terms of building a non-blocking client. It just doesn't have the required functions to allow that kind of usage.

One workaround, at least for DBD::mysql, would be to code a small threaded engine. We would use C-level threads, one for each connection, and the master Perl/XS module would communicate with them to simulate non-blocking behavior. This would not be something new, IO::AIO uses this trick with much success (see the impressive performance that you can squeeze out of projects like qpsmtpd, Perlbal, or DJabberd, all built on top of IO::AIO via Danga::Socket).

But we are back to the master-slaves approach of the current CPAN solutions, only now in-process so the communication between threads should be pretty fast and simple. And its specific to DBD::mysql. Its better but still not good enough.

I'll keep poking at this from time to time. I would like to have a solution that would only require DBI for this. My next item on the "things to try"-list is reading the code for DBD::Gofer. In itself, I don't see it as a solution, but I'm hoping to learn something that triggers a possible approach for this.

So for now, I'll keep using the master-slaves approach. While not perfect, it's good enough for now.

Update: also to check, the callback mechanism of DBI, as explained by Tim Bunce.