tl.dr; The migration went smoothly and I'm very happy with the service. I still have two things I need to do (SSL certificate and migrate my XEP-0114 external components), but for personal reasons I won't have the time to finish the migration until next month. So expect a small update when I finish those two.

For now, I'll just describe the service and my migration process, the good parts, the could-be-improved parts and other tidbits.

The Hosted.IM service provides a hosted XMPP service powered by a carrier-grade ejabberd XMPP server. The service was created and is maintained by Process One, the main driving force behind ejabberd development over the past years. I had the pleasure of working with Process One and Mickael in particular when we migrated the SAPO XMPP server.

The basic service is free (5 user accounts, a single domain).

The main roadblock preventing my migration was the couple of custom-made external components I wrote to interact with some web and TCP-based services that I currently host, so I needed proper XEP-0114 support. I asked Mickael for it and less than two weeks later, they delivered. The XEP-0114 support was officially announced yesterday on all non-free plans.

With this final roadblock lifted, I was ready to migrate my domain.

I've registered a new account, and added my domain, simplicidade.org. The first thing they ask you to do is to add a DNS TXT record to validate that you own the domain. I don't understand why they need this TXT records. The XMPP service requires you to add or update a couple of SRV records to point to the Hosted.IM XMPP servers. If you aren't the owner of the domain, you won't be able to update those records, so why ask for an extra DNS record? I hope they clear this process and remove this particular requirement. Or, if this is really something that they really need, add a FAQ explaining why its needed.

I also immediately updated my SRV records to point to them, using the provided examples. This turned out to be a mistake that you should avoid.

If you are migrating an existing domain, I strongly recommend that you don't update the DNS SRV records at first. You should first create the accounts on the new service, migrate the current users rosters and vcards, and only then switch the DNS records. This should be pretty obvious stuff, but I was eager to move and failed here.

The Process One support personal will accept a SQL dump of your rosters and vcards and load them up on the service at Hosted.IM. I was lucky because I was already using ejabberd with the SQL backend, so I only had to cleanup old accounts, dump the SQL database and send them the file.

This data migration process is unfortunately not documented yet. New users don't even know the possibility exists. I had to ask for it on Twitter to discover the possibility. I also don't know what other formats or other servers export files they support. So check with support before you decide to switch to figure out how you'll migrate the roster information.

As I said, I didn't prepare that part, so I had to scramble to dump the rosters and send them the SQL, so that my users didn't end up with a empty roster. Fortunately the support staff was awesome and I quickly had my SQL dump loaded onto the service.

After this was done, I closed the firewall for my old server C2S ports, and started up my XMPP client. I connected without any problems to the new service.

From start to finish, and even with all the discovery and learning the layout of the administration interface, it took me less than 2 hours to have my service migrated. Pretty good.

I then selected the cheapest plan, at €8/month. The payment system is not clearly explained at the site. It works as a prepaid system: you load your account with €nnn and they remove the amount you owe every month. You also get some bonuses if you load your account with large amounts. After spending 10 minutes wondering where the payment interface was (after you change plans, the interface appears in the account section; while on the free plan, it's not visible - this is suboptimal, a link in the Plans & Pricing tab, or near the costs values in the domain administration tab would be more helpful), I loaded my account with €100 and I got a €8 bonus.

Some services cost extra. For example, using your own SSL certificate is a €2/month, and connectivity to other IM networks costs €4/month. Unfortunately, these extra costs are not described in the service homepage. You have to register for the trial account, and then check the Plans and pricing tab inside your domain management admin page.

Aside: on the service homepage, if you select Plans & Pricing in the navigation toolbar at the top, the javascript scrolls down to the #pricing section but fails to update the page location, which makes sharing the direct link harder.

And this is were we stand right now: accounts, rosters and vcards were migrated successfully, and I was able to load my account with enough money to last me for a bit less than a year (I was not counting on the extra €2/month for the certificate).

The next step is creating my own certificate. This part of the process could be improved a lot. For technical reasons, you have to upload the certificate private key. But if that is a requirement (and it is, I understand that part pretty well), then they could save a lot of work to their clients if they just took care of all that: add the option to generate the key on their servers, and send me the Certificate Request file so that I can request a certificate from a CA that supports XMPP certificates (which are slightly different from HTTPS certificates, they require an extra extension). It would be helpful if they recommended a couple of CAs providing the service, but they do not.

In the past, the XMPP Software Foundation provided a free service of XMPP Certificates, but it was shutdown sometime ago. According to their page above, you can buy a XMPP certificate from StartSSL CA, but I'm still figuring out how to do this. It should be straightforward, the same process as a HTTPS server, and I'll update this article after I've successfully done it, but the StartSSL site lacks XMPP-specific information.

After I have that part done, I'll move my external components. Some of them are sub-domains of my main simplicidade.org domain and those should be straightforward.

Others use a completely different domain name. This is an unusual setup. I basically used my own ejabberd server as a XMPP router for some domains. I connected those domains as external components, and pointed the S2S DNS records to the ejabberd server.

The Hosted.IM does not yet support this mode of operation, but I again asked Mickael about it, and this unorthodox configuration should be supported very soon. Awesome.

All in all, a pretty smooth ride.

Twitter will live on as a mainstream platform, used by countless millions of regular every-day non-technical users for which these new rules are meaningless. They will suffer and question why their previous favorite client no longer works, while downloaded the latest official Twitter abortion of an interface. They will complain about it (their $Deity given Internet right), but will move on, and keep on using it.

But those of us more technically-able will probably look for a new partner, while we lick the stab wounds inflicted but our former mistress.

As always, when something wrong happens to a centralized system, the geek-gene arouses and looks for the greener pastures of decentralization. "Would it be good to have a Twitter without the central service?"

Apart of the love/hate relationship with centralized systems, of which the current cloud-fad is one mighty engine, I have to say that the only word who comes to mind about all this is dejavú

Almost-optimal solutions for this problem exist for decades. The first one I remember is good old multicast. Assuming unlimited address space, assign each person his own multicast group and let the routers take care of it. Its unpractical of course, but the architecture of any solution that emerges will have a lot of common points with this old codger.

But more recently some systems tried to tackle the problem. My pet favorite, XMPP, is one of them. We have per-user publish/subscribe that can be used to implement such a system. And we have a large network of XMPP servers already.

But XMPP has two drawbacks. The first is that the explosion of a message to all his subscribers is done at the source server. This means that when the Bieber opens his mouth and bleats, his own server would have to send his pantomime individually to all members of his flock. So even if a thousand of them were on the Google Talk network, instead of a single message, the Google servers would receive a thousand messages. I would be remiss if I didn't mention that there is a deferred extension to tackle this problem, but few in the community showed interest on it.

The second drawback is more perception than technical: over the years I felt that people don't see XMPP as Webby enough. Its a strange, dark technology, that few understand, and the passion for it by those who do, makes those who don't stagger.

But the pretty solution of a multicast network to deliver information, even in the XMPP approximation of a network of centralized servers, where each one acts as a aggregator for a part of the community, providing distinct local services for local people, has one glaring drawback: there is no privacy.

If I delegate one instance of my message to a Google aggregator that serves their clients, I can only hope that they will do the right thing and deliver said message to the right people.

If this requirement is soft for public messages, it becomes very strong for protected streams. If you want some assurances that your protected tweets don't end up in the wrong place (and I set aside the discussion about what kind of guarantees we have about that from Twitter itself in its quest to monetize your content) then you can't use this remote-server-side explosion but have to go back to exploding your messages individually to each of your approved subscribers.

There is an extra cost on resources for this kind of privacy.

So if you want to try and tackle the distributed Twitter problem, please remember that it is one of the most discussed architectures in our Internet history. It has been done before (or at least tried) several times, some with success, others less so. It doesn't matter if the next real-time micro-blogging tool will be based on Usenet, SMTP, RSS, XMPP, or the latest fad-du-jour. The basic problems are the same ones our fore- fathers tried to solve with multi-cast.

As I said, dejavú.

post scriptum rant, or why I need to support <aside> on my CSS: But is a decentralized system with some central aggregation nodes based on trusts workable? History has something to reminds us again: the usenet was the elite messaging tool amongst geeks in the early 90's (oh how I miss thee, trn). Then the AOL horde invaded and it became mainstream. The geek clan left, and later the usenet collapsed with spam, and now is a strange BitTorrent look-a-like.

The clan moved over to blogs and RSS. And those too were colonized, although not as thoroughly because that RSS thing is still a bit on the complicated side.

So the social networks were born, to hide all those pesky RSS feeds, add a dash of real-time, and place a pretty face on top of all of it.

So: do all decentralized systems evolve into spammer havens or centralized silos by sheer force of capitalism? A topic for another day.

Until we do know more about it, we can look through the Wave protocol draft spec, and notice that its built on top of XMPP.

At the same event, the new XMPP-powered mini applications where also announced. Each one is a XML file with all the HTML, CSS, and JS files packed together. Time to see how long it will take a XMPP desktop client with access to a WebKit view to implement this extension. It reminds me all the discussions about embedded applications we had in the API mailing list last year.

I'll keep it running in case you need a guinea pig for tests.

Very good news.

The AnyEvent::DNS module allows you to resolve DNS queries fast and in parallel.

To test this, I wrote a simple command line tool to check XMPP SRV records of a list of domains. Both server-to-server and client-to-server records will be checked and results printed out. All queries will be made in parallel.

The code is at my Ironman repository. Click the "Raw" link to download (sorry, no direct link, GitHub lacks permalinks to the latest raw version of a file).

But Kevin is not out of the XMPP client world. He and Remko announced a new cross-platform client called Swift.

Not much is known for now about Swift but I expect a bit more news over the weekend.

While we wait, I'll leave here my wish list for my next client. I don't know if Swift is going in this direction or not, this is just me thinking out loud:

• C/C++ minimal core: the core is responsible for XMPP connections and stanza parsing;
• HTML-based UI: all windows are HTML-based. If they choose to use Qt, they would be WebKit views. The DOM of those views would be extended with XMPP-oriented objects;
• JavaScript-based application: the IM logic is written with JavaScript, manipulating the XMPP DOM objects and the HTML-based UI;
• Plugins: all features should be written as plugins. I should be able to drop a directory with HTML, CSS, images and JavaScript files and hook that into the XMPP dispatcher. It should be possible to hook using XPath expressions, so that I can build new protocols with ease.

We can dream, right?

Good intro, solid presentation. Go, read.

I didn't have Keynote.app on the laptop where I wrote it so I decided to try S5 to write my slides.

The HTML version is great, and with a bit of syntax highlighter even the XML examples look good.

But the PDF version is very basic. I searched around for a better CSS to print S5 slides, but could not find anything. Maybe CSS is just too limited for decent printing control.

I don't know if I'm going to use it again. The lack of a decent PDF output is a big minus. On the plus side, I really like having the full power of HTML/CSS/JS inside my presentations.

As for the code examples, I wrote two:

• a basic HTTP-to-XMPP gateway: post stuff and send it via XMPP to someone;
• synchronize several processes to execute in a sequence (for example, restart a web farm, one web server at a time) using a chat room. Includes an auction (very simple, assumes everybody plays fair) to determine the final sequence.

All examples are written in Perl using the Net::XMPP2 framework. I made a few fixes to some of the Net::XMPP2 modules, you can find them inside the repository to make running the bots easier. I'm going to send Robin Redeker (Net::XMPP2 author) those fixes, so they should appear in a future version.

The blog post mentions the usual suspects: XMPP for signaling, RTP for transport and a H264/SVC codec. The implementation uses a browser plugin. Time will tell if this plugin will be bundled with Chrome (most likely) or bundled with Gears (would make sense for Google).

The Mac installer includes a meta-package with two packages: Keystone.pkg and the GoogleVoiceandVideo.pkg.

The Keystone.pkg includes the GoogleSoftwareUpdate.bundle that is installed at /Library/Google. Inside it has a GoogleSoftwareUpdateDaemon but so far I don't have it running on my system.

The GoogleVoiceandVideo.pkg is the real deal. It installs several things:

• two QuickTime components, Google Camera Adapter 0 and Google Camera Adapter 1;
• an Internet plug-in, googletalkbrowserplugin.plugin;
• an GoogleTalkPlugin.app support application.

After installation, it opened my GMail account and detected my iSight camera. I wasn't able to find someone to use it with so far, so I'll report back on the quality of the image. I expect a decent quality given the chosen codec.

I'm focusing on Codebits right now, so not much time to check out the XMPP part of this, but it seems to me that then reused the same signaling already used by the Google Talk client, nothing new there.

The big question for me is this: will other sites be able to use this plugin for their own video-chat features? I need to read the license more carefully.

As for Skype, it still has one advantage: I can use it without opening a browser on my Mac. But if I where them, I would start thinking about XMPP support and interoperability with Jingle, but thats just me.

Skype window of opportunity to be the leading XMPP client is shortening every day.

So get of your collective chair-interface-parts and send your ideas.

Thanks!

I also have a slot to talk about XMPP and this is where you can help.

I don't want to do another abstract talk about XMPP, but instead I want to write code that you would like to see.

So if there is some XMPP-based functionality that you want to see how you can get it done, ping me or email me with the high-level details.

I'll take all the projects I can cover in an hour and present my skeleton or even complete solution at the conference and discuss why it was done that way.

All the code will be available before the presentation so you can also follow along.

I spent an enormous amount of time there and I still have fond memories of that time, of the people, the stories, everything.

But access to Moosaico (and other MUD's or Talkers) is usually done with Telnet or using a client like TinyFugue or Atlantis. Keeping one of those open to be connected somehow never entered my normal workflow, so I drifted away.

Its amazing that something that I was hooked on 14 years ago is still alive in me.

The friday something happened that will pull me back in.

Another friend, Marcos Marado, commented that he was looking for a XMPP interface to his talker, Selva. I had an headache and didn't feel like working on what I was supposed to, so I took the challenge.

Later that afternoon, Rasputine was born.

Rasputine (or Ras, as Corto Maltese called him) is a generic Moo/MUD/Talker-to-XMPP gateway. You add a buddy to your roster and then you can use it to connect to that world.

For example, to log in to Moosaico, add moosaico@rasputine.simplicidade.org and then type //connect. Presto, the login window of Moosaico should be readily available. The Selva talker is available at selva@rasputine.simplicidade.org.

The code is still very young and feature-less. The copy that I run on my staging server already has per-user status messages and avatars, and next I plan to implement auto-connect (probably even XEP-0100 support).

For now, I'm just enjoying getting back to Moosaico.

Update: by the way, if you want to connect your own talker/MUD/Moo, ping me via XMPP or email and I can set you up. At least to try it out.

(via IMtrends)

I gave a presentation entitled WTF is XMPP? mostly centered around non-instant messaging applications of XMPP. I think it went well, the room was pretty full and I talked a lot with people after the event which plan to use XMPP in the short term.

The organizers did a wonderful job, and I want to thanks Alcides to push me to write the presentation.

Update: cool, the presentation was featured on SlideShare homepage.

Kevin, Peter and Remko are hard at work for O'Reilly to brings us the XMPP missing book.

Entitled "XMPP: The Definitive Guide", it should hit the shelfs in 2009.

Shameless offer: if you need technical reviewers, you know my JID :).

It seems to be MUC-based, not PubSub. I hope to see a future version with PubSub in the future.

I used the Net::XMPP2 Perl module, in particular the Net::XMPP2::Component class. It uses the AnyEvent async framework, which in turn support the EV library , giving you all the love of kernel polling.

Until the Net::XMPP2 author releases a new version, you should use my own copy if you plan on doing external component work (check the component-reply-with-from branch). Net::XMPP2 is widely used for bots, but not that many components, and some methods need some love to work properly in a component environment.

The HTTP part was done using the excellent AnyEvent::HTTP class. I'm using my own version, which includes a bug fix to the http_post function (on its way to release 1.03 of AnyEvent::HTTP) and adds support for HTTP::Request to http_request. I hope to see this included in the main AnyEvent::HTTP distribution but I still need to update the documentation.

The rest is just glue code.

These are some suggestions that bot authors can use to increase your XMPP presence easily. I'm going to stick to easy stuff only, most of it widely deployed.

The first thing that you need to understand is that you can have different per-contact presence information. You can switch the status message and the avatar for each of your contacts. Its called directed presence (basically include the contact JID in the to attribute, but you can always check the spec for directed presence).

You can use this to provide a personalized status message for each person on your roster. If your service has a notion of context, you can use the status bar to show the current context. For example, imagine that you have a ticketing system bot. You type "working on TICKETID", and your bot can change the status message to "Working on: TICKETID - TICKET_TITLE (URL of ticket)".

Optionally you could update the status each five minute with the elapsed time since you started working on it.

You can also switch the avatar. Maybe switch to a red background, meaning that you are busy doing something, or putting it in another way, the bot is telling you that he thinks you are working on something.

Other stuff you should do is to provide an alternative representation of the information you send textually. Twitter (when it used to work...) was a great example of this: every tweet you received by IM, included an Atom entry with the most important information of the tweet. This allows client to improve their interaction with such services, by using a more appealing interface.

But Atom entries are not yet standardized by the XMPP Foundation (there is some effort on Atom over pubsub that could be used as an example though). On the other hand, Data Forms are a standard, and more, they provide interaction possibilities.

You could send a "New ticket" notification, including a data form with the structured information, and a select-box with the possible next steps. The user would see the form, and submit back their decision.

Still in the subject of messages, you should know that there are several types of messages. You have chat, headline and normal (strangely enough the default). There are two more but not important in this context. You should use the correct one depending on the type of message you are sending.

If your system is just sending notification, please use the headline type. It makes it clear to clients that they are not chatting with you, just notifying of something that happened. For example, a smart client can react to those messages differently based on your status: in away, xa (extended away) or dnd, a client can keep a log of new headlines and show it like a email client when you return, without interrupting your workflow, probably using a badge with "You have X pending headlines".

The final recommendation: respect the user status. If the user is dnd, are you sure you should send him anything? Why not change the status message to "You have 5 messages pending. Send 'pending' or go to URL to see them"? Maybe even change the avatar color. You should not disturb someone in dnd. I would also say that away and xa are off limits but that's just me.

On the other hand, you should send "Hey there, welcome back! You have 5 pending messages. Send 'pending' to see them or hop on to 'URL'." when the presence changes to chat or available (not a real <show> value, just lack of <show> value. See the valid values of <show> and their meanings in the spec).

Have fun implementing your bots.

Thats a lot of new victim^H^H^H^H^H^Husers to the XMPP network.

This, and the fact that my Google Alerts on XMPP is now showing about 10 hits per day (from less than 10 per week just 18 months ago), and I would say that XMPP is turning out to be one hot technology at the moment.

Update: given that the news came up via the official Facebook developers site, some people have IM'ed me asking why the question mark in the subject. Well, I'm waiting to see if they will open federation or not. Using XMPP is more than allowing Facebook users to use their favorite XMPP client to chat with the other Facebook users. Without open federation, this is just another private IM network and I think we have enough of those.

So the question mark is there until a answer is found for "Will Facebook enable open-federation over XMPP?"

One problem was that open-source servers did not implement PIP/PEP. Recently this has changed, with both Openfire and ejabberd having decent implementations. Other servers, though, are still MIA.

Also of note, GTalk does not supported it yet. This is a bit of a problem due to his visibility in the XMPP community.

So if you do implement it in your client (and I think you should, really), it seems that you have to keep using the current protocols for avatar (for better or worse XEP-0054 plus XEP-0153) and server-side storage (XEP-0049).

There is something that servers could do to improve this situation though. There is a small change that would give client authors even more good reasons to implement PIP/PEP today.

If private storage and vCards are implemented on top of PIP and PEP, we could have the best of both worlds.

From the point-of-view of current clients, nothing will change: they will still use XEP-0049 to set and get their private storage nodes, and they will still use XEP-0054 to publish and retrieve their vCard. But at the same time, each private storage item would also be available in a PIP node, and the user vCard would be available in a PEP node.

This would allow PIP/PEP-capable clients to announce support for vcard-temp+nofity namespace in their disco response, and get instant notifications of vCard changes. Also, announce storage:bookmarks+notify and clients that only support version 1.0 of XEP-0048 will immediately support version 1.1.

This could be a easy way to increase supply of nodes, to see if we can reach critical mass sooner, in the move to a pubsub-world.