« April 2007 | Main | June 2007 »

May 31, 2007

Google Gears

Google announced Google Gears, a browser extension that allows web applications to have a offline-mode available.

Right now, most recent browsers are supported with two omissions: Safari (on which the team is working on - my bet is expect this with Leopard and the new, more powerful, WebKit according to the developer (7:00), you can build a Safari plugin from the SVN code but another post mentions that it will only works with the nightly builds of WebKit), and Opera (which is not even mentioned).

The two cool features I like about this is that they provide you two standard APIs: a SQL-based storage layer, using SQLite (with the fts2 full-text search extension which is a nice touch); and a worker-thread pool for long running jobs.

The other interesting thing mentioned is a strong focus on synchronization, which is for me the most important aspect right now when building any kind of application. I do believe that if you are starting a new application (web-based or desktop-based) and you don't think about synchronization between data-stores, the lifetime of your app is small and quickly getting smaller.

Synchronization is one of those things that is easier to do if you plan your data-store from the start with sync in mind, it will be a lot easier.

Taking this idea forward, and gluing it with SamePlace, the next step for me is to think how can we put both of these APIs inside a XMPP client.

Think XMPP-powered Grid applications, with a web interface, capable of working offline. Think of personal-assistant bots that can negotiate meetings with other personal-assistant bots.

update: the code is open source, it "almost" compiles for Safari.

update 2: Safari is supported, you just don't get a binary for download, and you need to compile from source.

update 3: Safari appears to be supported only with nightly builds of webkit so it seems that my first idea that it would be a Leopard only thing might be right after all. They mention that, with Safari, the Google Gears is both a plugin and a Input Manager. I heard somewhere that Input managers would be gone in Leopard so something doesn't feel right.

May 30, 2007

Impressive

Nothing more to say, really...

May 25, 2007

Developments on the SIP vs XMPP front

If you are interested in the interoperability between XMPP and SIP, these two articles might be of interest:

May 24, 2007

About scalability and performance

Memorable quote:

There is a computer science mantra (one of many) stated by Donald Knuth as; "Premature optimization is the root of all evil." I've argued for a long time that defining "premature" for a particular situation is what separates a senior and a junior engineer.

Go read.

May 22, 2007

Module of the Day: Text::Unidecode

For all of those moments where you would write:

s/áéíóúàèìòùãõâêîôûç/aeiouaeiouaoaeiouc/g;

Now you can do the right thing with Text::Unidecode.

May 19, 2007

Strange dreams, fun software

At the last geek lunch, Nuno I talked about some script to index mail using KinoSearch.

Sometime this week, Nuno asked me for a reference to that tool, and I just could not find it. I'm now pretty much sure that I was dreaming about it.

But the idea nagged me for a couple of days, and today I broke and took Mail-tools and KinoSearch for a ride.

The result is maildir_indexer.

Right now it works. It indexes about 100 messages per second on my laptop. Searching is instantaneous. I'll index about 10Gb of mail tomorrow to have better numbers. The index size is about 1/3 of the data.

This was a quick hack and it shows. Feel free to send patches, but look at the future/todo list first.

May 16, 2007

I love corporate meeting requests

My latest one:

  • 32 recipients;
  • marked as 'High Priority'.

I love people who mark all outgoing emails as high priority...

May 14, 2007

The empire strikes back

Microsoft is moving.

Whatever the outcome of this, the landscape of software development will never be the same.

May 07, 2007

It's Monday...

... so start with a laugh.

May 04, 2007

Dear Lazy Web: distributed election algorithms

Dear Lazy Web, I need to implement a distributed election algorithm over an mostly-reliable group communication channel.

Any pointers to papers or software that already does this would be mostly appreciated. I'll post my own findings here also.

Kisses,

May 01, 2007

Sync power

There as been a lot of talk about offline modes for Web applications lately, and this path is getting closer to what I see as post-web-2.0 generation of applications.

Someone will eventually coin a term to describe these apps, but I think the following features will be part of them:

  • offline mode: there are those who believe that universal, always on, connectivity is just around the corner. I just might, but personally I think that they live in a nice dream and will eventually wake up. Even if you get there, there is no reason to always be online. Sometimes is good to unhook yourself;
  • sync services: this one is a direct result from the previous one. We are not talking about access to your data hosted on a web site somewhere with a REST/SOAP API, we are talking about full-blown sync, where you can add, edit, delete content offline that will sync to a group of interested parties when you happen to be online;
  • content push: right now, we are in the era of pull. The inefficiencies of using polling will eventually catch up with us, and we will see that in a notification-based world, push is the only way to go;
  • global pubsub semantics: subscribing RSS feeds is a great concept, you only get what you are interested in. Pubsub will only make it more sane on the bandwidth-side of things. We might all have FTTH soon, but that doesn't mean that is ok to waste it.

This might seem strange, but my biggest beef with the Web-2.0 generation is a totally irresponsible disregard for the environment. Here I am, sitting on a Intel Core Duo processor, burning watts, and I'm suppose to outsource my CPU to a distant server. Yeah, that makes perfect sense, right? Its a total waste, and it can't last.

Having a local application in sync with a remote server on the off chance you might be without your laptop/PDA/whatever is a much greener alternative, and makes use of all these local resources you have with you. Those 100Gb disk drives on your latptop can be put to a good use. Think them as the fastest storage component in a global hierarchical storage management system, where S3 and friends will probably be your long-term storage. Imagine having your operating system telling you that you have 100 terabytes available of storage, and letting him make the decision on which files to keep in you local cache.

The necessity of universal peer-to-peer sync services in the new generation of applications will see a flourish of activity in conflict resolution and merge algorithms. The good news is that the last 2 to 3 years, projects like darcs, Mercurial and Monotone, have been working on those issues, so if we leverage the work done by distributed SCM systems, we should have a good head start.

The dynamics of peer-to-peer networks has also a good set of solutions to stand on, thanks to all the peer-to-peer sharing networks, and more recently to Bittorrent. I don't expect to see a direct mapping onto those protocols, but surely the ideas that are behind them will be used in the universal sync network.

There is no reason to throw away current applications. In fact I believe that most NG applications will still be made in Python, Ruby and Perl. The advantages of scripting languages is immense in terms of fast prototyping. Also, given that we will run them locally, "fast" will take on another meaning. A web-app-local-environment, will give you a generic package format, which will allow you to drop the the local version of, for example, Basecamp. A common API will give you access to basic services like storage, databases and network. And each operating system will probably add there own APIs to access his features.

The is also no reason not to use the current HTML/JS/DOM model as the preferred "view". After all, it is probably the most universal and cross platform toolkit we have. Those standards are not standing still, and the richness of the controls available will only increase with time.

Finally there is the question of network protocol that will power all this. How will these applications talk to each other. How will my calendar accept and decline meeting invitations based on my free time. I'm a bit biased on this topic. I do believe that XMPP provides most of what we need. In a world of social relations, your buddy list already contains the trust network that you can leverage for authorization. End-to-End encryption will keep it all reasonably safe. And the current pub-sub specification, while not scalable enough right now, can be easily improved to become a global universal notification network.

The last paragraph might be wishful thinking, but it doesn't cease to amaze me the brain power that goes into things like Comet, to get that illusional always-on pipe to the application, to push real time content, when you could simple have a XMPP client inside the browser and expose the protocol via a small DOM.

In my next installment, I'll talk about Xiclets, a small proposal to do just that: build a common DOM for Javascript and HTML applications to use XMPP as a grid protocol.