« July 2007 | Main | September 2007 »

August 31, 2007

Gentlemen, start your Bittorrent clients

Quote of the week:

Sometimes I think God put video content guys on the planet to make the music guys look progressive and visionary.

nuf said.

iPod vs MP3

While talking to Alcides Fonseca about my kids, I remembered that I wanted to share something with you: my 3 years old knows the difference between iPod and MP3. And what's more, it wasn't me.

He picks up a Creative Labs thingie and calls it MP3, but both mine and Rosario's iPod are truly iPods.

Cool.

Next step: being able to distinguish between Python, Perl and Ruby...

I wonder...

In a site I manage that requires user registration, 5.4% of the users use as password a combination of 4 digits.

I wonder how many of those match their PIN numbers of their credit/debit cards.

push @wish_list, "High Performance MySQL, 2ed "

I guess we'll have to way a bit, but given the pedigree of the author list, this will most definitively be the MySQL book to buy.

August 30, 2007

Maildir curiosity

/me chuckles

By some strange measure (basically looking at the table of supported mailbox formats across open source IMAP servers, mail clients, and local delivery agents available here), maildir is more popular than the legendary mbox.

But I went to that page to checkout the new dbox format. I'm intrigued.

August 23, 2007

KDE and Git

KDE is looking at Git, at least in a theoretical "how would we structure this large project in Git"-kind of way.

There was a lengthy response from Linus, with a couple of gems inside:

I certainly agree that almost any project will want a "central" repository in the sense that you want to have one canonical default source base that people think of as the "primary" source base.

But that should not be a technical distinction, it should be a social one, if you see what I mean. The reason? Quite often, certain groups would know that there is a primary archive, but for various reasons would want to ignore that knowledge.

Right on. Git can model SVN centralized model if that is what makes sense in your organization, and you can still have all the benefits of Git.

If you are thinking if Git (or Mercurial even) is for you, and you are a SVN user, check the Git for SVN users document.

August 22, 2007

Zee pain

So after the latest laptop cleanup (format, install), I forgot to install AntiRSI...

Yep, i can't feel my right arm. So I will be working at half-speed for a day or two.

As usual it could not come in a worst time, but there some things that you don't mess with.

XMPP-based notification of incoming emails

A nice tool that you can use to receive XMPP notification on incoming mails.

Some facts:

  • uses inotify to detect new emails, so Linux only;
  • Maildir-based deliveries only;
  • Perl script.

My personal attempt at this was not as simple as this one. The current idea is based on qpsmtpd and ActiveMQ but its still in the "back of the brain"-stage.

August 21, 2007

I need some sleep...

I just wrote this:

  $value =~ s/(\s)\s+/$1/g; # Highlander filter for white-space

Yeah, sleep deprivation does that to you...

August 20, 2007

XMPP plumbing

This has been on my mind for quite some time, but during an email reply to Adam Nemeth today, I found myself putting it into words I like.

Jabber will not succeed outside geekdom by selling the IM part of XMPP. That's my gloom point of view.

I think that XMPP can be pushed to the clients as an enabler, as infrastructure, on top of which you build several services, in particular notification, but in general a always-on-presence, that all the usual services (email, blogs, social networking sites) can leverage to make their own services better.

In short, XMPP is for me the pipe to the desktop that all the other services need to improve their services:

  • I send an email, important to me, and I add an header saying to my server that I want a immediate notification of a response. My email server uses that to trigger a XMPP pub/sub notification telling me that a reply is ready in my inbox;
  • Someone posts a comment on my blog, I'm notified;
  • Someone adds me as a buddy on some social site, I get a notification;
  • Someone invites me to a meeting, and the meeting request can travel through XMPP, check my own free time (using a negative presence agent that I have running connected to my calendars), and reply back;
  • Google indexes a page that mentions my blog, I get a notification.

After you have basic XMPP plumbing in your desktop, adding IM a simple add-on to increase the value of the plumbing.

So my view, is: sell XMPP as a service to other services, IM will flow naturally from that.

So far I had modest success selling this, so I'm not promising it will be easy.

Family project

So my kids are 1.3 and 3 years old now, and the oldest one loves a computer game that the place we have breakfast at on weekends has, based on a touch screen interface.

It has a set of games, like painting black and white layouts, filling the spaces with colors, basic memory games (6 cards down, with pictures, pick two alike to make them stick), and differences between pictures games.

I just got a quote for 15" TFT touch screens. Including a fan-less Via-based computer, with 500MB ram, and 30Gb disk, and a license of Windows XP, the whole deal comes about 1300€...

Hmms... I wonder if I can make the same games using HTML and the canvas stuff? I would build a full-screen web-kit, they would have a home-page for games, photos, what not...

Implementations details

Two things to make your week start ok:

I needed that. The rest of my week, work related, is not going to be easy or fun.

August 18, 2007

What's wrong with current Wikis

Clarification: it seems, reading the comments, that people think that I'm talking about Wikipedia and their choices. I'm not. I'm using Wikipedia as an example as how this system could work and the benefits. Now back to our regular program....


I've always had a love/hate relationship with Wikis.

On one hand, I like the perceived easy updating with the direct access provided by those edit links on all the pages. But on the other hand, using a browser to drive the whole experience is a limiting factor.

My hate pushed me to think about writing Kiwi, something I talked about years ago. Kiwi is not a Wiki would be the defining phrase of the project.

The current web-based wikis fell very limited mostly due to the use of a browser as the main editing tool. There are other alternatives to keep a Wiki and collaborate with others.

One of them is distributed version control systems. Lets play a game, and design Wikipedia on top of one of the current crop of such systems. For purposes of demonstration, I'll use git, but you could also choose mercurial, darcs, monotone, even svk if you are masochist enough.

In a git-powered-Wikipedia-world, the main Wikipedia site would host a branch, that eventually is converted in the site wikipedia.org you all know and love. And you would still see those nice edit links all over. But when you click on them, and edit a page, you would be committing your changes to the repository. From the perspective of the user, nothing has changed.

But now, anybody can pull that repository, edit to their hearts content and then push the results back. Wikipedia foundation would probably prefer to pull from registered repositories (everybody could register a repository) and they would probably only accept the changes if they don't conflict, but that is just mechanics.

But now, all the Wikipedia is available on your local disk, and there is where the fun starts.

You no longer have to use a web-based editor, or some hack to edit text-areas in a remote editor. You can now see lots of Wiki editors appear. They will be able to give you (if that is what you want) WYSIWYG editing. They will give you a local index that you can search. Even offline, you can search, read, edit every page. And then push back your results.

Even if you do like to edit a wiki in a web-based editor, you could still have a local application/webserver that provides that edit environment. Imagine a Backpack-like interface (the new one, just released is much much better) to your own personal, local wiki. But now, you can share it easily with others.

You can also start using GPG or other public-key cryptography system to create a trust level of edits. Unsigned edits could still be accepted (although I have zero problems with mandatory GPG signing of wiki edits: privacy and anonymity are two different things, and while the first must be a right, the second one should not), but now you can look at a author and look at his web of trust. Wikipedia key signing parties would flourish.

If Wikipedia starts getting to big to download it all (probably already is), you would start to see subsets of Wikipedia, as intermediaries to the big one. Anybody could create their own subset, maybe focused on a topic, or area of knowledge, and you could just pull from that one. But all the changes you push to those middle-mans would flow back to the top. And given the nature of most distributes version control systems, your changes are in fact cryptographically signed, so they can't be changed by nobody without being noticed.

Another outcome of the switch to such a system is that you no longer have to push back all your changes. You could grow a departmental wiki side-by-side with Wikipedia. In fact, you could create a application that pulls from several wikis, including personal, work, project you follow, and see them integrated in a single application. In the same way you today subscribe a RSS feed, you could now subscribe to a wiki, and have all that shared knowledge available at your fingertips. If you know VoodooPad, imagine having your pads side-by-side with public wikis. (On a side-note, in my original design of Kiwi, each page is a Kiwi, each Wiki is a Kiwi tree, and the set of Wiki trees you subscribe to is called an Orchard. Page templates are called seeds. Talk about an analogy going off tracks...).

The mentality that is preventing us to move forward in a collaborative world, is that people are creating a very strong link between the concepts of collaboration and web-based system, like you can't have a collaborative system without the second one. That is wrong on so many levels. The web should be just one of the mechanism in which you share the knowledge, but starting the design of a wiki based on the assumption that the only interface will be a web browser severely limits his potential.

Its time to stop building version control systems on top of SQL databases, and move to the real thing. Its time to let the content of Wikis be free, easily available, to be used in ways not restricted by the web environment.

Angie

ejabberd is getting better and better every day. You can look over the developments in the last 6 months following the Process One blog:

The last one is the one. This will make it trivial (for those fortunate enough to be using a publish-subscribe-type system already as one of the tools in their business) to publish via XMPP business-side events, like blog posts, comments, uploaded photos, new classified ads in your area, job offers, and much much more.

With that out of the way, maybe we could try and put a XMPP client inside our browsers, switch to a real push-based network, and stop wasting our time with "AJAX-poll-like-crazy" or "Comet-half-baked-everything-is-http" solutions to asynchronous event notification in a large scale.

DBI and Async loops

Most of my time, I program inside async event loops like Danga::Socket or POE.

Accessing DBI inside those loops is not a straight forward thing. Most solutions involve forking worker threads and using pipes to communicate between my script and those workers. There are a couple of components for POE that do most of the work out of the box, like POE::Component::EasyDBI, but still, it feels a lot like an hack.

For Danga::Socket loops, I've been working with two "simple" solutions:

  • split the work between sync and async tasks, using disk-based storage to move work from one side to the other;
  • use HTTP-based REST web services.

There are two more solutions that might work now. The first is the amazing DBD::Gofer. I haven't played with it yet (look over the Tim Bunce presentation at CPAN to get an overview) but it simplifies the client side of things that it might just be possible to tweak it into a async DBD driver. The DBI API would have to be slashed a bit, I don't think it has a async version.

Gofer is nice, but will still require a HTTP server for the Gofer servers. And if I have a HTTP server, I might prefer to have a higher level API that can also group some queries in a single call, some of them could even involve a transaction that Gofer does not support.

The other solution is to use Gearman. Its fast, and seems to have all the niceties for scale (multiple workers, multiple managers). But it is not reliable, at least not until the client decides to make it so with code.

All in all, I think both solutions are good, and you can even use Gofer for some things, and Gearman for others. Heck, you can even use Gearman as back-end for Gofer.

For now, I think I'll try Gearman, it seems less work, and I'm extremely lazy. But I'll get back to Gofer soon. I would love to see a asyncronous DBI API, and DBD::Gofer might just be the door.

Startup performance of DBIx::Class

In a project I was working on, I had some performance problems to startup a DBIx::Class schema with about 75 sources. It took about 19 seconds to startup.

After a quick thread in the mailing list, the startup time is now 2 seconds.

The two-part solution is this:

  1. move all your load_components() into a common class and use that class as the base for your sources;
  2. use the schema provided load_classes(), its very fast. If you need per-source tweaking, do it afterwards looping over Schema->sources().

Many thanks to the dbix-class mailing list, in particular mst and Hartmaier Alexander for the tips in the right direction.

August 17, 2007

Tip: use growlnotify for long running Terminal tasks

If you have Growl installed, and you use the Terminal.app a lot, then make sure you also install the growlnotify command line tool.

This will allow you to send Growl notifications from the command line. The most useful script I have, that I use constantly is this:

#!/bin/sh
#
# Runs script, and prints a notification with growl when it finishes
#

$*
growlnotify -m "Script '$*' completed" -s "Background script notification" &

My version is called n, just the single letter n.

This allows me to do:

n scp server:some_big_file .

and a Grown notification will appear when the process terminates.

Update: a couple of updates with great suggestions from Ranger Rick, Tim Bunce and Ruben Fonseca. The new version below should also work on Linux systems, using libnotify. It was updated to deal with arguments containing spaces and we keep the exit status of the command intact. Also we assume exit code 0 is success, all others mean some failure has occurred. Thanks to all.

The Linux version needs checking, I didn't have a Linux server with libnotify, and used Ruben suggestions blindly. Please leave a comment if it doesn't work, specially the detection code. You might need a -h in there.

The full script (download):

#!/bin/sh
#
#  Runs script, and prints a notification with growl when it finishes
#
# Written sometime in 2006, posted 2007/08
#
# With Tips from Ranger Rick, Tim Bunce and Ruben Fonseca
#

# Run the command, including arguments with spaces
"$@"
status=$?

# decide which status to use
if [ "$status" == "0" ] ; then
    result="completed"
else
    result="FAILED ($status)"
fi

# decide which notifier we have
env growlnotify -h > /dev/null 2> /dev/null
has_growl=$?
env notify-send -? > /dev/null 2> /dev/null
has_libnotify=$?

# notify the user, growl or libnotify
if [ "$has_growl" == "0" ] ; then
    growlnotify -m "Script '$@' $result" -s "Background script notification" &
elif [ "$has_libnotify" == "0" ] ; then
    notify-send "Script '$@' $result" "Background script notification" &
fi

# exit with the original status
exit $status

Update 2: typo in "has_libnotify" corrected. BTW, I'm using env to check availability of a command because which (my first choice) returns the same exit code if the program exists or not, at least in Mac OS X.

Update 3: Thanks to Ruben Fonseca, the above script (and the download version) now work correctly on Linux with libnotify.

August 15, 2007

Simplification

Imagine this:

  1. you have a set of standards that have grown complex over time;
  2. people all around are using variations of the theme, simpler and effective;
  3. you decide that you should also simplify things on your side.
  4. what do you do?

Of course! You set up six committees to simplify your protocols. Because we all know that committees and simplification come hand-in-hand, right?

OMFG...

I'm sick, I could not stop giggling with this.

Fortunately I'm not alone.

August 11, 2007

Wow...

Does this mean that they are bankrupt?

August 09, 2007

iGTD stuff

I'm enjoying my iGTD experience. I think there is a lot of good stuff in there, and with some MailActOn stuff and some mail rules, I've been able to use very effectively as a Inbox for everything.

I still miss a easy way to track external tickets, but maybe we can work around that with the Pro version and some plugins.

Update: Ok, found it! Just select all the projects... Stupid me. Now on to step 3 of my list: I have this list of tasks for a set of projects, I want to filter per context.

One thing that I don't like at all is the query capabilities. I miss a view that combines Project and Contexts.

I would like to see something like this:

  1. select one or more projects to focus on (usually one);
  2. check some box to include sub-projects also;
  3. select one context;
  4. apply.

This would give me all the actionable tasks that I have in those projects in that context. So far, I haven't found a way to do it with iGTD.

See this screenshot: if I have 12 tasks in that project, including sub-projects, I would expect that, if I select that project, I would see the 12 tasks. But no, I only get to see the ones in that project exactly, not a single one from the kids. That is the correct behavior if I expand the project to show sub-projects (iGTD even updates the count to 2 in that situation, correctly). Only the task list does not adjust properly for collapsed projects.

Fortunately.... The data is inside a SQLite database... hmmms... :)

HTML5, Micro-formats, and the progress tag

Just now, in the uf mailing list, there was a message by Alexandre Van de Sande about a Developer Works article about HTML5.

It seems that HTML5 has a set of semantic tags with lot of potential, and some of them cross into (until now) micro-format territory.

I skimmed through the Developer Works article and one particular element caught my attention: <progress />.

From the article:

The progress element represents the state of an ongoing process, like the progress bar in a graphical user interface (GUI) application. For instance, it can show you what percentage of a file is downloaded or how far you are into a movie. This progress control says that a download is 33% complete:

 <p>Downloaded: 
    <progress value="1534602" max="4603807">33%</progress>
 </p>

The value attribute shows the current state of the operation. The max attribute shows the total amount toward which the progress is moving. Here the element indicates that 1,534,602 bytes out of a total 4,603,807 bytes have been downloaded.

You can display indefinite progress bars by omitting the max attribute.

You should use the JavaScript language to dynamically update the progress bar as the operation continues. Statically, this element isn't very interesting.

First, apart of styling-control, I don't see why dont browsers gives us a standard file upload dialog (probably modal for the tab), stating which file I'm currently uploading, the percentage, and the speed, all data that the browser has access to. Maybe there is a Firefox Plugin, maybe.

Second, this is a good tag, but the last part, about dynamically update the progress bar with Javascript, I wonder how exactly would it work: will it require the current system of server-side tracking of uploads and AJAX pooling, or will the browser itself generate Javascript events with the progress?

I've got to find time to read the about HTML5 and the WHATWG.

Display resolution

At work, I use two displays: the internal 17" 1680x105 of the Macbook pro, and a external 20" 1680x1050.

The problem is that the DPIs of each monitor are different: the external has 3" more at the same resolution.

This makes me almost never use the 17" for work stuff, and stuff it mostly with long-lived tasks or log tailing windows.

Now I'm trying something different. I've lowered the resolution of the 17" to 1440x852 (stretched) and that makes the 17" much closer DPI-wyse to the external one, and I can have real work windows there.

System Preferencesscreencapture002

Will see how it goes. I don't know if the fuzziness that this stretching causes will make it a very livable experience. Time will tell.

I can only hope that Leopard and its resolution independence will allow me to set both monitors to the same DPI auto-magically.

August 08, 2007

Apple keynotes and the Apple TV

Is it just me or isn't it a bit stupid that I cannot see Apple Keynotes in a Apple TV.

I can sit through gigabytes and gigabytes of YouTube trash, but looking at Apple Keynotes, noooo....

sighs

HTML Rich Editors

This last week I went around shopping for a HTML rich editor, those "little" JS things that learned in Hogwarts (Hi, pfig! ;) ) some spells to turn normal text-areas into Word look-alikes.

Anyway, this is my short list right now, in no particular order:

  • Asbru Web Content Editor: fully supports Safari, commercial but inexpensive. Supports tables;
  • YUI Rich Text Editor: fresh out of the oven, not tested yet, but YUI always makes my short list;
  • ExtJS Lightweight HTML Editor: I already use ExtJS so this also makes the list, but it seems very limited, to be investigated;
  • TinyMCE: this has been my choice on other occasions but Safari support is lacking a bit.

At the moment I'm biased to the first one, just because it supports Safari and Table editing. But I might stick with ExtJS simpler editor if it has all the feature I really need (mostly bullets).

Apple stuff

Busy day, so not much time to write. Highlights from yesterday Apple announcements:

  • F$%#$%Q&#$%! Airport Extreme got GigabitEthernet now. I bought mine 4 weeks ago! Anybody wants one, good old Airport Extreme base station? Tested? The reason I would like to have gigabit: shared disks, which this base station supports;
  • The new keyboard is beautiful, but I will have to wait until I can try it out. I don't know if I will like the feel. I'm very glad that they upgraded the wired version to USB2, so that I can use the ports to sync iPods;
  • if the new iMovie has direct YouTube upload (maybe a feature of Quicktime itself?), why doesn't iPhoto allows me to upload to Flickr? Guess I have some place to spend my $30 (see below);
  • Numbers: I'll download the demo and see what the fuss is all about. It wont replace die hard Excel fans like Rui (and I understand why), but it might be enough for simple people like me. But, I'm willing to bet that the analysis features of DabbleDB (my favourite Hype2.0 site) will leave both Numbers and Excel in the dust;
  • the iPhoto Web Galleries: shiny. Pity about the size;
  • the new Macs (iMac and Mini): if I wanted a desktop Mac, I would probably buy a Mac Pro. The difference between a desktop and a laptop is expandability, and the new models are both limited in that regard (I can't see why the iMac does not offer the option of a second hard-drive. Can't they find the space behind those huge displays?). The Mini would be on my wish-list if they made a version with a basic Mac OS X Server edition. That I would buy. They could add a couple of inches and make it dual-drive RAID1 by default: nirvana. (Update: having said all that, the new iMac ads make me want to rush out the door and buy five of them).

Some other odds and ends, and rants:

I don't understand people who like bluetooth or RF keyboards:

  • I would hate to be out-of-batery in the keyboard;
  • I don't want the bother of changing or recharging batteries in a keyboard;
  • the fact that a wireless mouse is extremely useful (because mice where made to be moved around and the cable gets in the way) does not translate to the keyboard, something that you hope that does not move around (rubber feet, essential feature of a keyboard).
  • given Macs poor USB port count, even the two extra ports in the keyboard are extremely useful, even more by the fact that they are very accessible;
  • the $30 difference is the price of several great looking, extremely useful apps by independent developers out there, which will probably make a bigger difference on your productivity than one less cable.

Still on the USB port count, the iMac specs page is funny:

Total of five USB 2.0 ports: three ports on computer, two ports on keyboard.

Err... Pity that the Keyboard that has 2 USB ports is the wired keyboard, and that one takes one of the other 3 ports, so let's just say:

three USB ports. If you use the included keyboard, you get two more! If you decide you wanna look cool using our bluethoot keyboard, you just get those three ok?

There. Much better.

On the department of things I would like to see in a future opportunity:

  • the ability to buy iLife and iWork as a package or as standalone apps via iTunes, to keep on abusing the 'M' in iTMS. I couldn't care less if they called anything else, but the infrastructure is there, just use it. I don't need the entire package, but I would update some of the apps;
  • there is this technology inside every mac called Bonjour, don't know if you hear of it. Apparently Apple has not. Why cant I do a slideshow over local lan, that other users can watch in real time? Or why can't I share a Pages/Keynote/Numbers session? Like Subethaedit? Room to grow.

That is all.

August 03, 2007

Payments

If you want/need to receive money from your customers online, you now have another option: Amazon Payments.

250 pages of docs... That will take time.

Let see this stick...

First of all, if you like the music you are listening to right now, that you downloaded from somewhere, and if you have no plans on paying for it, you should be ashamed.

I would like to see this stick though:

Tracking down suspected file-sharers may soon become more difficult throughout the EU. In an advisory opinion released shortly after the Offenburg decision, an advocate general for the European Court of Justice ruled that ISPs are not required to disclose subscriber information in civil infringement cases.

via Ars.Technica.

August 01, 2007

Love / Hate relationship with GTD

I've read the book, adjusted some of the things mentioned to fit me and my personal style, but I've always fallen short of nirvana.

My "inbox" is very distributed. I have to check several ticketing systems for several projects I participate, on top of all the things that arrive via email.

I'm also very command line oriented, and I still write a lot of code, sprinkled with TODOs and FIXMEs.

What I wanted is a tool that collects all that, with my help, and keeps me up-to-date. And a Pony, it seems.

So far all the this is not available. All the GTD helper applications (current favorite is iGTD) that I've tried are:

  • single user: no easy way to delegate a task in a structured way, and receive updates on it;
  • no semi-automatic external inputs: if we had some sort of plugin system, iGTD could be extended to watch specific tickets on specific ticketing systems, as my own tasks, and allow basic interaction (update, close);
  • task dependencies: iGTD sort-of has this, but doesn't track dependencies across multiple contexts.

The first one could be solved with a XML representation that is sent via email. A mail rule on the receiving side would integrate it in the other person GTD app, probably placed in the inbox for acceptance / rejection, with a checkbox "send updates to the sender".

If the email configured for a project is a mailing list, several people in the project could be in the loop.

The second one is very complex because there are several ticketing systems out there. In the same way that Blogging APIs are more or less converging to two or three standards, it would be great if a simple REST API could be developed that would allow integration with desktop apps.

In the meantime, GTD apps could allow external plugins to integrate with those systems.


Right now, I'm giving iGTD a try. Previously I was using todo.txt but it was not enough for some of my needs, like hierarchic projects and tasks dependencies.

I have to take tickets from external systems and place them in my personal iGTD, but that is only a F6 away, for now.

Lets see if I survive a week with this...

Refreshing views

I like this article. Competition rules.

Now start flaming away, I know YOU want it... Hmms... I need a badge, "Flames accepted" or something, there is a badge generator somewhere...