Notes
Building simplicidade.org: notes, projects, and occasional rants
email github facebook twitter flickr vimeo

Sync power

There as been a lot of talk about offline modes for Web applications lately, and this path is getting closer to what I see as post-web-2.0 generation of applications.

Someone will eventually coin a term to describe these apps, but I think the following features will be part of them:

  • offline mode: there are those who believe that universal, always on, connectivity is just around the corner. I just might, but personally I think that they live in a nice dream and will eventually wake up. Even if you get there, there is no reason to always be online. Sometimes is good to unhook yourself;
  • sync services: this one is a direct result from the previous one. We are not talking about access to your data hosted on a web site somewhere with a REST/SOAP API, we are talking about full-blown sync, where you can add, edit, delete content offline that will sync to a group of interested parties when you happen to be online;
  • content push: right now, we are in the era of pull. The inefficiencies of using polling will eventually catch up with us, and we will see that in a notification-based world, push is the only way to go;
  • global pubsub semantics: subscribing RSS feeds is a great concept, you only get what you are interested in. Pubsub will only make it more sane on the bandwidth-side of things. We might all have FTTH soon, but that doesn't mean that is ok to waste it.

This might seem strange, but my biggest beef with the Web-2.0 generation is a totally irresponsible disregard for the environment. Here I am, sitting on a Intel Core Duo processor, burning watts, and I'm suppose to outsource my CPU to a distant server. Yeah, that makes perfect sense, right? Its a total waste, and it can't last.

Having a local application in sync with a remote server on the off chance you might be without your laptop/PDA/whatever is a much greener alternative, and makes use of all these local resources you have with you. Those 100Gb disk drives on your latptop can be put to a good use. Think them as the fastest storage component in a global hierarchical storage management system, where S3 and friends will probably be your long-term storage. Imagine having your operating system telling you that you have 100 terabytes available of storage, and letting him make the decision on which files to keep in you local cache.

The necessity of universal peer-to-peer sync services in the new generation of applications will see a flourish of activity in conflict resolution and merge algorithms. The good news is that the last 2 to 3 years, projects like darcs, Mercurial and Monotone, have been working on those issues, so if we leverage the work done by distributed SCM systems, we should have a good head start.

The dynamics of peer-to-peer networks has also a good set of solutions to stand on, thanks to all the peer-to-peer sharing networks, and more recently to Bittorrent. I don't expect to see a direct mapping onto those protocols, but surely the ideas that are behind them will be used in the universal sync network.

There is no reason to throw away current applications. In fact I believe that most NG applications will still be made in Python, Ruby and Perl. The advantages of scripting languages is immense in terms of fast prototyping. Also, given that we will run them locally, "fast" will take on another meaning. A web-app-local-environment, will give you a generic package format, which will allow you to drop the the local version of, for example, Basecamp. A common API will give you access to basic services like storage, databases and network. And each operating system will probably add there own APIs to access his features.

The is also no reason not to use the current HTML/JS/DOM model as the preferred "view". After all, it is probably the most universal and cross platform toolkit we have. Those standards are not standing still, and the richness of the controls available will only increase with time.

Finally there is the question of network protocol that will power all this. How will these applications talk to each other. How will my calendar accept and decline meeting invitations based on my free time. I'm a bit biased on this topic. I do believe that XMPP provides most of what we need. In a world of social relations, your buddy list already contains the trust network that you can leverage for authorization. End-to-End encryption will keep it all reasonably safe. And the current pub-sub specification, while not scalable enough right now, can be easily improved to become a global universal notification network.

The last paragraph might be wishful thinking, but it doesn't cease to amaze me the brain power that goes into things like Comet, to get that illusional always-on pipe to the application, to push real time content, when you could simple have a XMPP client inside the browser and expose the protocol via a small DOM.

In my next installment, I'll talk about Xiclets, a small proposal to do just that: build a common DOM for Javascript and HTML applications to use XMPP as a grid protocol.