Alex and Adam asked me to elaborate on the tools I used to implement the Tarpipe XMPP gateway.
I used the Net::XMPP2 Perl module, in particular the Net::XMPP2::Component class. It uses the AnyEvent async framework, which in turn support the EV library , giving you all the love of kernel polling.
Until the Net::XMPP2 author releases a new version, you should use my own copy if you plan on doing external component work (check the component-reply-with-from branch). Net::XMPP2 is widely used for bots, but not that many components, and some methods need some love to work properly in a component environment.
The HTTP part was done using the excellent AnyEvent::HTTP class. I'm using my own version, which includes a bug fix to the http_post function (on its way to release 1.03 of AnyEvent::HTTP) and adds support for HTTP::Request to http_request. I hope to see this included in the main AnyEvent::HTTP distribution but I still need to update the documentation.
The rest is just glue code.
The current crop of the XMPP-based bots is pretty basic. They provide an online presence in the XMPP network, and you interact with them by a command-line-style interface.
These are some suggestions that bot authors can use to increase your XMPP presence easily. I'm going to stick to easy stuff only, most of it widely deployed.
The first thing that you need to understand is that you can have different per-contact presence information. You can switch the status message and the avatar for each of your contacts. Its called directed presence (basically include the contact JID in the to attribute, but you can always check the spec for directed presence).
You can use this to provide a personalized status message for each person on your roster. If your service has a notion of context, you can use the status bar to show the current context. For example, imagine that you have a ticketing system bot. You type "working on TICKETID", and your bot can change the status message to "Working on: TICKETID - TICKET_TITLE (URL of ticket)".
Optionally you could update the status each five minute with the elapsed time since you started working on it.
You can also switch the avatar. Maybe switch to a red background, meaning that you are busy doing something, or putting it in another way, the bot is telling you that he thinks you are working on something.
Other stuff you should do is to provide an alternative representation of the information you send textually. Twitter (when it used to work...) was a great example of this: every tweet you received by IM, included an Atom entry with the most important information of the tweet. This allows client to improve their interaction with such services, by using a more appealing interface.
But Atom entries are not yet standardized by the XMPP Foundation (there is some effort on Atom over pubsub that could be used as an example though). On the other hand, Data Forms are a standard, and more, they provide interaction possibilities.
You could send a "New ticket" notification, including a data form with the structured information, and a select-box with the possible next steps. The user would see the form, and submit back their decision.
Still in the subject of messages, you should know that there are several types of messages. You have chat, headline and normal (strangely enough the default). There are two more but not important in this context. You should use the correct one depending on the type of message you are sending.
If your system is just sending notification, please use the headline type. It makes it clear to clients that they are not chatting with you, just notifying of something that happened. For example, a smart client can react to those messages differently based on your status: in away, xa (extended away) or dnd, a client can keep a log of new headlines and show it like a email client when you return, without interrupting your workflow, probably using a badge with "You have X pending headlines".
The final recommendation: respect the user status. If the user is dnd, are you sure you should send him anything? Why not change the status message to "You have 5 messages pending. Send 'pending' or go to URL to see them"? Maybe even change the avatar color. You should not disturb someone in dnd. I would also say that away and xa are off limits but that's just me.
On the other hand, you should send "Hey there, welcome back! You have 5 pending messages. Send 'pending' to see them or hop on to 'URL'." when the presence changes to chat or available (not a real <show> value, just lack of <show> value. See the valid values of <show> and their meanings in the spec).
Have fun implementing your bots.
Came across this just now: Using Facebook Chat via Jabber
Thats a lot of new victim^H^H^H^H^H^Husers to the XMPP network.
This, and the fact that my Google Alerts on XMPP is now showing about 10 hits per day (from less than 10 per week just 18 months ago), and I would say that XMPP is turning out to be one hot technology at the moment.
Update: given that the news came up via the official Facebook developers site, some people have IM'ed me asking why the question mark in the subject. Well, I'm waiting to see if they will open federation or not. Using XMPP is more than allowing Facebook users to use their favorite XMPP client to chat with the other Facebook users. Without open federation, this is just another private IM network and I think we have enough of those.
So the question mark is there until a answer is found for "Will Facebook enable open-federation over XMPP?"
My favorite XEPs of last year where PIP (XEP-0223) and PEP (XEP-0163). These two made Publish-Subscribe (XEP-0060) trivial to implement on XMPP clients, and pave the way for rich-presence clients.
One problem was that open-source servers did not implement PIP/PEP. Recently this has changed, with both Openfire and ejabberd having decent implementations. Other servers, though, are still MIA.
Also of note, GTalk does not supported it yet. This is a bit of a problem due to his visibility in the XMPP community.
So if you do implement it in your client (and I think you should, really), it seems that you have to keep using the current protocols for avatar (for better or worse XEP-0054 plus XEP-0153) and server-side storage (XEP-0049).
There is something that servers could do to improve this situation though. There is a small change that would give client authors even more good reasons to implement PIP/PEP today.
If private storage and vCards are implemented on top of PIP and PEP, we could have the best of both worlds.
From the point-of-view of current clients, nothing will change: they will still use XEP-0049 to set and get their private storage nodes, and they will still use XEP-0054 to publish and retrieve their vCard. But at the same time, each private storage item would also be available in a PIP node, and the user vCard would be available in a PEP node.
This would allow PIP/PEP-capable clients to announce support for vcard-temp+nofity namespace in their disco response, and get instant notifications of vCard changes. Also, announce storage:bookmarks+notify and clients that only support version 1.0 of XEP-0048 will immediately support version 1.1.
This could be a easy way to increase supply of nodes, to see if we can reach critical mass sooner, in the move to a pubsub-world.
Last February, Google's Android Team shocked the XMPP world by having the gall of saying that they where moving to a Binary encoding of their GTalk API in the M5 release.
Of course this sent shock waves through the XMPP community, not because they where totally wrong, but because they didn't have the vision to solve it.
We of course understand that they have their hands full (of vapor, some bad mouths would say...) getting the platform in shape for the late 2008 release, and the XMPP.. err, sorry, GTalk protocol is not a top priority.
Well, today Android users around the world can rejoice because once again, the XMPP Standards Foundation has risen to the challenge and unveils to the world the solution to the "verboseness" problem, by introducing the breakthrough Binary XMPP protocol.
This has been in development for quite some time, taking long hours of (sometimes) heated discussion. The goal is to have something that nobody could ever accuse of being verbose (the symbol count is extremely low, and the meaning is clear from the start), but at the same time remain compressible for those extreme cases where bandwidth is at a premium (we tested, and we have code to prove it: a Binary XMPP stream can be compressed to 2% of its original size). We know of clients that live in five-sides-shaped-buildings that, with their hard 9600 baud limits, will be the first to use this.
Its been a great ride, and it feels me with immense joy to be able to put my small signature as a co-author of this spec. Of course, my part is extremely small when you have the documentation genius of Peter Saint-Andre, and Fabio Forno strict guidelines and requirements, as co-authors. Its been a pleasure to work with both on this. And I should also thanks the important Kevin Smith contributions.
And now time to rest. SRV records are up at simplicidade.org domain, and Fabio's CM is running also.
PS: and yes, now you know why the prolonged silence around here.
PPS: for those who have moved on, the code is also available on GitHub.
When I worked at SAPO, I started using a MSN account to keep an eye on the pyMSNt transport that we have there. Eating your own dog food and all that.
So now is the time to stop using it. I'll unsubscribe the MSN account from the Gateway, and I'll no longer be available on the MSN network. You can add my XMPP address as a buddy: melo@simplicidade.org.
Right now, you have a lot of open and decent XMPP servers out there: SAPO Messenger, Google Talk, Jabber.org and a lot of others. So there's really no excuse not to have an account on one of them.
So hop on to the clients list and join the fun side.
If you want to try some agent-love, then let me introduce you to Alter Ego, your friendly pluggable agent.
Alter Ego is a XMPP agent, where everything is a plugin. You can clone the current version with git (recommended, easy to keep track of new features and fixes) or use the tarballs.
Right now, you can find it running under my own JID, so yes, I trust it enough to run it on my main account.
The only plugin I wrote is a User Location (XEP-0080) updater. With all the cool geo toys (like Fire Eagle) around, I wanted to play a bit with location-aware services. So the current UserLocation plugin does a very simple thing:
So you do need a PEP-enabled server like ejabberd 2.0 or OpenFire.
Also, nothing prevents you from using other sources of location like a GPS. For example, I'll probably include something like this perl-GPS simple example in a future version of the plugin.
You can look at the roadmap to see what I'll be working on, or you can subscribe any of the zillions of feeds that github provides. I usually keep all the development of each plugin in a separate topic branch, so you can track a specific development easily.
If you want to pitch in, I recommend that you get an account at github and fork the project. Github has some nice workflow tools that make it very easy to track each other work. If you need a github invite, ping me via XMPP.
One of the things I mentioned in my XSF Membership application was my endless curiosity about negative priorities in XMPP.
When most people talk about XMPP, they focus on the Instant Message network thats built on top of the XMPP federated network. Some also focus on the aspects of application automation. But even this second group rarely dwells into the wonderful and mystic world of the negative priority.
For those out of the loop, the <presence> stanza you send at the start of the session to signal your availability has some child elements. The most common ones are <show> that tell the other users your status (like away, busy, and the rarely respected do not disturb) and <status> that holds your claim to fame on the "I'm smart and witty in 100 chars or less"-category.
The other standard attribute is <priority>. Priorities range from -128 to 127. If you omit it, your priority is set to 0. Servers use the priority to decide which of your multiple connections (usually called resources) will receive messages without a specific resource in the destination to attribute. So if you send a message to melo@shakespeare.lit/a_resource, then the message is delivered specifically to the resource a_resource, but if you send (as you should on the first message) to melo@shakespeare.lit then the server decides which resource will get the message based on the resources priorities. You can find the all the dirty details in RFC 3921, in particular section 11.1, paragraph 4.1.
The second part of the last paragraph is where things become interesting for negative priorities. If you send a <message> to my bare JID (without a resource), and the only resources available have a negative priority, the server will treat you as an offline contact and store the messages in offline storage.
So negative priority resources work as a non-chat connection, that you can use for other purposes, like automation or application-to-application communication.
The usual bots that you see online have their own JIDs and you talk to them over chat-based interfaces. But with negative priorities you can have a different kind of bot, one that connects with your own JID, but with a negative priority. To distinguish these two types, I'll call the second one an agent. It acts on your behalf inside the XMPP network.
Imagine that you write a agent that has access to your calendar and task list. Given that this agent also has access to your roster, you could use that as an access control mechanism to allow your buddies agents to consult your calendar, negotiate meeting requests based on a free time calendar, or even delegate a task to you.
Or another one: you have a agent running on each computer that you own, and you configure it to export certain directories. Again, you can use your roster group structure to define who has access to what. You can re-arranje your buddies in your main roster, the agent receives the roster push and immediately adjust his ACL.
The possibilities are pretty much endless and this is a area where very little work and study has been done until now.
But there are some road blocks ahead. The first one is that most clients out there don't deal properly with negative priorities. If a buddy of mine only has negative priorities, my IM client must show him as offline. Most clients right now, don't do that.
Worse: when I was trying this over the weekend, I was able to crash some clients (I could see them logging off and login back in continuously).
But that's ok. There was no reason (and right now, there still isn't a good one) for client authors to worry with this. Nobody was using negative priorities. And I'm sure most authors will fix this as time goes by.
Next up: communication with your agents.
So there are all this ingredients floating around:
The glue of course will be Perl, and the first step is already working. This program will print my location once per second:
use strict;
use warnings;
use lib 'lib';
use GPS::NMEA;
my $gps = GPS::NMEA->new(
Port => '/dev/cu.GPSmart-SPPslave-1',
Baud => 9600,
);
while(1) {
my($ns,$lat,$ew,$lon) = $gps->get_position;
print "($ns,$lat,$ew,$lon)\n";
}
This is a pure copy-and-paste version of the script in the GPS::NMEA manual page, it just works (after you set the correct Port), and it was my first version.
I did get some warnings, so be sure to download and install the latest development version of perl-GPS (You must specifically tell cpan to do it).
Next steps: hack some bot with Net::XMPP2 to publish this information.
The Open AIM effort is a welcome step by AOL. But is it in the right direction?
If you are thinking about writing a XMPP Transport/Gateway using these specs, you should read carefully the Terms and Conditions page. There are two issues I can see:
And if you though these could be worked around, then let me point you to the "Are there any restrictions on what I can build?" FAQ entry in the General section so that your hopes go bye-bye:
Although we have removed many restrictions on usage and development, we still do not permit developers to build Open AIM applications that are interoperable with other IM networks. (Multi-headed applications are now allowed). Please refer to the Developers License Agreement for additional details.
So no, you cannot write a AIM transport with this.
So yes, our best hope is still that the XMPP client-to-server gateway that AOL was playing with sees the light of day.
And yes, in an ideal world it would be a server-to-server gateway, but I think it takes a lot of little steps to get there.
This Open AIM is just not one of those steps.
If you want to get your hands dirty in ejabberd goodness and have a more erlang'iang brain than my own, here is a simple request: hack and slash into the PEP code, and allow the use of an ACL to decide which users (JIDs) can publish to the node. Bonus points if you limit the nodes/namespaces to which I can publish to.
I need to authorize a trusted external component to publish into PEP nodes hosted at my domain.
The final goal is to bridge Fire Eagle to User Location (XEP-0080). It goes like this:
In an ideal world, I should be able to request permission to publish to the JID PEP node, you would get a nice little message with that, and you would give you permission. But support for pubsub in clients is still very poor, so we have to use this ACL-in-server-hack.
Mind you that this required cooperation of server operators to authorize publishing to PEP nodes. Again, its the best we can do right now.
Also, an ideal Fire Eagle API would also notify me with Web Hooks that the presence of users I subscribe to was changed.
(hey, and if you are reading this and are currently at ETech 2008, please talk to any Fire Eagle developer you can get your hands on and ask them politely to support Web Hooks).
Last week there was some chatter about Google decision to rename the XMPP API in Android to GTalk API, and dropping the XML-based wire protocol for something binary.
The reason that the Android team gave for this was: XML is too verbose.
First, let me state that I agree that the API should be changed to GTalkAPI if its no longer a compatible XMPP API. This leaves the name open for a future real XMPP API, so in this limited way, it makes total sense to rename it.
Second, "XML is too verbose" seems to me a quick and dirty excuse, and I would prefer to have a real reason for this. A couple come to mind:
I don't know much about mobile platforms to say if the CPU and/or battery usage is an issue, but I wouldn't be surprised if it were. On the other hand, if the new binary protocol doesn't offer some sort of encryption, well, it won't really be an option, right?
But the most interesting discussion that this change will trigger is about XMPP wire-level protocol.
The upper layers of XMPP should be XML. Our specs are written in XML and I only see advantages on doing so. But this does not mean that the wire protocol must remain as XML. We must understand that XML is just one of the possible representations of the XMPP protocol. It's the one we choose in our documentation, because its the one we humans find easier to read and understand. But that's it. In the end it will be machines that read and parse and interpret the XMPP protocol, and the wire representation should be the most efficient to the machine. Having a XML console is nice, and it will always be there, even if we change the wire representation.
There are alternatives to a UTF-8 human readable version of XML, and one of the most likely to succeed is Efficient XML Interchange (EXI). It remains to be proven, as Robert points out, that EXI (or any other binary encoding applied to XMPP) is more efficient than the current XML-over-zlib-over-TLS.
But making justice to the X in XMPP, we can extend XMPP to support these alternative encodings, using an existing standard, XEP-0138. There is already a proposal for a EXI encoding for XMPP using that same XEP. So experimenting with different encodings can be done within the standard, and that is good.
So we can see that we don't have to use XML as a wire representation and still be XMPP compatible. The next issue is: is the problem with mobile platforms and XMPP related to the XML representation or the protocol itself?
Both Robert and the people participating in the discussion on the standards mailing list (several threads, but this is probably the most interesting one, specially after the change of subject), seem to think that the main issue is the very verbose nature of the protocol, specially with <presence> stanzas.
Also mentioned is the fact that XMPP is usually implemented over a TCP connection and that we don't have a good, quick and efficient way to recover a session after a dropped TCP connection.
So maybe XMPP is not very effective in the mobile world right now. But one thing is clear to me: a lot of people in the XMPP community are knowledgeable and even experts in that field, and so I'm sure we will see further specs to optimize the XMPP usage in a mobile environment.
But getting back to the XMPP wire representation. I really hope to have something more efficient than the current XML representation. And I don't say this because of the mobile world. I want this on the server-side.
More and more, servers are splitting themselves in standalone components: a frontend connection manager, a couple of core XMPP routers, and a set of external components, each dealing with specific features like file transfer, multi-user chat, or publish/subscribe.
And these components must talk to each other, and process a lot of messages. (as a side note, I once tested a commercial XMPP server, doing 500k messages per second). Even right now, I see servers doing tens of thousands of messages per second.
So given this traffic volume, making the overhead per message the lowest possible is a worthy goal, and if that means dropping the XML representation and adopting a new binary one, I for one will welcome the change, and the new binary overlords.
There is something in the air, because we now have two big IM providers flirting with XMPP. A couple of weeks back we could login to the AIM/ICQ network using XMPP. The trial seems to be over though.
But today, Mickaël Rémond tells us about Yahoo! Live experiments with XMPP.
Very cool.
Welcome aboard!
xmpp.oscar.aol.com:5222 seems to work. Login with your AIM/ICQ credentials.
Via Mickael and jdev.
Update: the service has been on and off as expected from an unofficial test service. The AOL engineers did a pretty good job for a alpha version.
So far this works:
ICQ_UIN@aol.com or AIM_SCREENNAME@aol.com;Things that I could not make it work:
All in all, its a great first version, and I can only hope that AOL starts talking about this publicly, because I think all the XMPP community is eager to help out.
Right now, we only have client-to-server via XMPP, so any XMPP client can connect to the ICQ/AIM servers but requires a ICQ/AIM account. Of course, the final destination is server-to-server connectivity, so that I can chat with AIM/ICQ buddies without a ICQ/AIM account.
When GTalk appeared, it took them about 6 months to open up server-to-server, so its normal to show up with client-to-server and move up to server-to-server.
So lets give them some time.
Congrats to all at AOL. Welcome!
After the Google/AOL agreement in late 2005, speculation regarding the interoperability between AIM network and GTalk network was rampant.
A screenshot of a new GMail client shows some AIM integration.
Unfortunately, the image shows "Sign out of AIM" which means that they seem to be doing a multi-platform client, and not XMPP federation. That is to say: you can login with your AIM credentials and talk with your AIM friends, inside the GMail interface, but you cannot use your GTalk account to talk to AIM buddies.
Of course, this is just a screenshot someone took somewhere. Their camera could be of the Photoshop brand. I do hope so.
If this is really what's going to happen, then the IM silos are still holding up.
A nice tool that you can use to receive XMPP notification on incoming mails.
Some facts:
My personal attempt at this was not as simple as this one. The current idea is based on qpsmtpd and ActiveMQ but its still in the "back of the brain"-stage.
This has been on my mind for quite some time, but during an email reply to Adam Nemeth today, I found myself putting it into words I like.
Jabber will not succeed outside geekdom by selling the IM part of XMPP. That's my gloom point of view.
I think that XMPP can be pushed to the clients as an enabler, as infrastructure, on top of which you build several services, in particular notification, but in general a always-on-presence, that all the usual services (email, blogs, social networking sites) can leverage to make their own services better.
In short, XMPP is for me the pipe to the desktop that all the other services need to improve their services:
After you have basic XMPP plumbing in your desktop, adding IM a simple add-on to increase the value of the plumbing.
So my view, is: sell XMPP as a service to other services, IM will flow naturally from that.
So far I had modest success selling this, so I'm not promising it will be easy.
As I said on my previous post, I'm looking over the documentation of the Google Gears (GG) stuff. I read it all yesterday at night, and though about it.
The things I like about GG are pretty simple.
The first one is that you can write your applications to take advantage of it if present, but work the same if not. Its not a all-or-nothing approach, its more of a progressive enhancement thing. It does require discipline, and I would imagine that projects like Jamal or TrimJunction (you can find more here) will soon start appearing everywhere.
The second is latency. If I can cache rarely changed stuff on the client, like list of friends in a social networking site, things that I use all the time in my UI, the latency of the interface is much lower, and therefore the experience much better.
The third is a very decent system for parallel programming on the web. The Erlang fans out there will like the design "share nothing, message passing" system behind the WorkerPool stuff in GG. I do plan to play with this a lot.
Finally, as a co-owner and developer of a e-commerce web site, any processing we can push to the client means less servers we need do deploy centrally and that means less capital investment. And that is definitively a good thing.
There are a couple of things I want to try out in the next few days.
The first one is to write a todo list manager. Its the most simple system you can build and it will allow me to use most of GG functions. the system will seamless synchronize todo items between a Catalyst-powered web app and the offline stuff in GG.
The second one is to try and run a JsJaC instance inside a WorkerPool. This would allow me to run a asynchronous XMPP client inside any web application with ease, making it possible to use the XMPP as push-style communication channel between back-end applications and front-end UI.
This last one is not a end in itself. JsJaC is using polling to connect to a XMPP server. But it allows me to think about a future XMPP DOM standard, available in XMPP clients where we could drop small applications, exchangeable between clients.
But that is last bit is for another brain dump in the near future. :)
Google announced Google Gears, a browser extension that allows web applications to have a offline-mode available.
Right now, most recent browsers are supported with two omissions: Safari (on which the team is working on - my bet is expect this with Leopard and the new, more powerful, WebKit according to the developer (7:00), you can build a Safari plugin from the SVN code but another post mentions that it will only works with the nightly builds of WebKit), and Opera (which is not even mentioned).
The two cool features I like about this is that they provide you two standard APIs: a SQL-based storage layer, using SQLite (with the fts2 full-text search extension which is a nice touch); and a worker-thread pool for long running jobs.
The other interesting thing mentioned is a strong focus on synchronization, which is for me the most important aspect right now when building any kind of application. I do believe that if you are starting a new application (web-based or desktop-based) and you don't think about synchronization between data-stores, the lifetime of your app is small and quickly getting smaller.
Synchronization is one of those things that is easier to do if you plan your data-store from the start with sync in mind, it will be a lot easier.
Taking this idea forward, and gluing it with SamePlace, the next step for me is to think how can we put both of these APIs inside a XMPP client.
Think XMPP-powered Grid applications, with a web interface, capable of working offline. Think of personal-assistant bots that can negotiate meetings with other personal-assistant bots.
update: the code is open source, it "almost" compiles for Safari.
update 2: Safari is supported, you just don't get a binary for download, and you need to compile from source.
update 3: Safari appears to be supported only with nightly builds of webkit so it seems that my first idea that it would be a Leopard only thing might be right after all. They mention that, with Safari, the Google Gears is both a plugin and a Input Manager. I heard somewhere that Input managers would be gone in Leopard so something doesn't feel right.
If you are interested in the interoperability between XMPP and SIP, these two articles might be of interest:
There as been a lot of talk about offline modes for Web applications lately, and this path is getting closer to what I see as post-web-2.0 generation of applications.
Someone will eventually coin a term to describe these apps, but I think the following features will be part of them:
This might seem strange, but my biggest beef with the Web-2.0 generation is a totally irresponsible disregard for the environment. Here I am, sitting on a Intel Core Duo processor, burning watts, and I'm suppose to outsource my CPU to a distant server. Yeah, that makes perfect sense, right? Its a total waste, and it can't last.
Having a local application in sync with a remote server on the off chance you might be without your laptop/PDA/whatever is a much greener alternative, and makes use of all these local resources you have with you. Those 100Gb disk drives on your latptop can be put to a good use. Think them as the fastest storage component in a global hierarchical storage management system, where S3 and friends will probably be your long-term storage. Imagine having your operating system telling you that you have 100 terabytes available of storage, and letting him make the decision on which files to keep in you local cache.
The necessity of universal peer-to-peer sync services in the new generation of applications will see a flourish of activity in conflict resolution and merge algorithms. The good news is that the last 2 to 3 years, projects like darcs, Mercurial and Monotone, have been working on those issues, so if we leverage the work done by distributed SCM systems, we should have a good head start.
The dynamics of peer-to-peer networks has also a good set of solutions to stand on, thanks to all the peer-to-peer sharing networks, and more recently to Bittorrent. I don't expect to see a direct mapping onto those protocols, but surely the ideas that are behind them will be used in the universal sync network.
There is no reason to throw away current applications. In fact I believe that most NG applications will still be made in Python, Ruby and Perl. The advantages of scripting languages is immense in terms of fast prototyping. Also, given that we will run them locally, "fast" will take on another meaning. A web-app-local-environment, will give you a generic package format, which will allow you to drop the the local version of, for example, Basecamp. A common API will give you access to basic services like storage, databases and network. And each operating system will probably add there own APIs to access his features.
The is also no reason not to use the current HTML/JS/DOM model as the preferred "view". After all, it is probably the most universal and cross platform toolkit we have. Those standards are not standing still, and the richness of the controls available will only increase with time.
Finally there is the question of network protocol that will power all this. How will these applications talk to each other. How will my calendar accept and decline meeting invitations based on my free time. I'm a bit biased on this topic. I do believe that XMPP provides most of what we need. In a world of social relations, your buddy list already contains the trust network that you can leverage for authorization. End-to-End encryption will keep it all reasonably safe. And the current pub-sub specification, while not scalable enough right now, can be easily improved to become a global universal notification network.
The last paragraph might be wishful thinking, but it doesn't cease to amaze me the brain power that goes into things like Comet, to get that illusional always-on pipe to the application, to push real time content, when you could simple have a XMPP client inside the browser and expose the protocol via a small DOM.
In my next installment, I'll talk about Xiclets, a small proposal to do just that: build a common DOM for Javascript and HTML applications to use XMPP as a grid protocol.
I'm no Ruby expert but a friend of mine was asking me how to write a simple XMPP bot using Ruby, and I had seen a couple of recent posts announcing a simple ruby library.
So after installing RubyGems (I had a total virgin Mac OS X stock install of Ruby), I did the usual:
sudo gem install xmpp4r-simple
That did not work well. One of the dependencies, rcov, does not install in my system. The error message is something like this:
gcc -fno-common -g -Os -pipe -fno-common -pipe -fno-common -pipe -fno-common -I. -I/usr/lib/ruby/1.8/universal-darwin8.0 -I/usr/lib/ruby/1.8/universal-darwin8.0 -I. -c callsite.c
callsite.c:121: error: parse error before 'event'
callsite.c: In function 'coverage_event_callsite_hook':
callsite.c:131: error: 'klass' undeclared (first use in this function)
callsite.c:131: error: (Each undeclared identifier is reported only once
callsite.c:131: error: for each function it appears in.)
callsite.c:136: error: 'mid' undeclared (first use in this function)
callsite.c:141: error: 'node' undeclared (first use in this function)
callsite.c: In function 'cov_install_callsite_hook':
callsite.c:162: error: 'RUBY_EVENT_CALL' undeclared (first use in this function)
make: *** [callsite.o] Error 1
To solve this, I did the following: first I manually downloaded the latest version of rcov from the rcov site. Doing the classic ruby setup.rb dies with the same error, but there is an alternative:
sudo ruby setup.rb all --without-ext
This works. Basically it skips installation of a C-based extension to the rcov package that makes the package a lot faster. Given that I don't intend to run coverage tests in ruby, I don't mind skipping this.
At this time, gem install xmpp4r-simple still tries to install the dependency of rcov although we already have rcov installed. So we skip the dependencies:
sudo gem install --ignore-dependencies xmpp4r-simple
This works, and now we can play.
ejabberd just reached 1.1.3, and you can download new fresh installers from the download page.
Of particular interest to me is the new Mac OS X Intel installer that allows me to have a ejabberd running on my Macbook.
With this, that makes three different XMPP servers now, that I run on a regular basis: ejabberd, Wildfire, and DJabberd. Each one has good and bad points. Lets see a couple of them.
Wildfire has the best admin web interface by far, and for small and medium size deployments (up to 3 or 4k online users) should work just fine. Adding plugins and most of the reconfiguring is done without service interruptions, has a decent Multi-User Chat (MUC), file transfer proxy, pubsub, and a beta gateway for other networks (MSN, Yahoo, AIM, and ICQ). All in all, pretty good package, and it's the one I use on my personal server right now. The only downsides for me are memory usage (he likes memory), and the fact that you cannot support more than one domain.
Ejabberd is the enterprise-class large scale deployment king. He keeps on working even when you throw at it indecent amounts of traffic. It has file transfer proxy, MUC, pubsub (although a bit out-of-date) and other goodies. The web interface is not the most beautiful but works and has most of what you need.
Ejabberd is also the only one of these three that can be configured as a cluster. Just add nodes and distribute your clients between them. Simple and just works.
These two are the ones you should look at first if you need a Jabber server for your company. They are actively developed and supported.
But if you like Perl and want to have fun, you can't beat DJabberd. You can easely modify any part of the server, it's all plugins, and there is still a lot to be done. There is no web interface, and only a handful of modules, but it works.
Use it if you need a very simple XMPP server or if you need to integrate with a already existing site or community.
All in all, I think that we have very mature XMPP servers out there. So, pick you poison and join the revolution.
Twitter.com is becoming a world wide chat-room or IRC channel, where two or three people have a conversation twitting their messages.
Crazy. Get a room already!
Yay! The Jabber Software Foundation is now the XMPP Standards Foundation.
I think this is great news.
Of interest: recently in a mailing list I subscribe to, there was an impromptu exchange of instant messaging addresses. I was very very happy to see that the majority had a Jabber address, mainly using GTalk service. This was not a geek list, by the way.
Some people I know are starting programming with Jabber/XMPP.
The classical book on the subject, "Programming Jabber (Programming)" (D.J. Adams), is a out-of-date, but is still good to browse for basic protocol notions, some of those are still the same. The entire book is also available online now.
The best book right now is probably "Jabber Developers Handbook (Developer's Library)" (William Wright, Dana Moore), but that is also a bit out-of-date.
So bottom line, there isn't a up-to-date book on the subject, so join the mailing lists at jabber.org and ask away...
For those in Perl-land, my suggestion would be to use the POE::Component::Jabber module, it's simple to use. I don't use the others in CPAN.
If you need help with Jabber programming, my Jabber ID is at the top of this page, left-hand side. Feel free to poke me.
The Jabber community lost a dear friend this week. Peter Millard, long time Jabber friend, and Exodus author, passed away after a struggle with cancer.
I didn't knew him personally, but the quality and breadth of his work for this community make him a dear friend to all of us.
Peter has written about this loss.
My deepest condolences go to his wife Christina, and is daughter Zoe.
Technorati Tags: jabber
A quick note to point out that version 0.3 of libjingle is out. As far as I can see, the major change is the inclusion of a lite version of the media component used by Google Talk.
Technorati Tags: google, googletalk, jingle, libjingle, xmpp
Jive Software announced Wildfire 2.5.0 just now. They are now 100% XMPP compliant. Congrats! They are the second server with source available to make that claim, after ejabberd last December.
I was still running an old Jive Messenger 2.3.1 so I decided it was time to upgrade. Installing a Wildfire is so easy that I never upgrade. I just export the users from the old server, do a fresh install of the new server, tweak two or three settings, and import the users. If you are upgrading from a Jive Messenger server, edit the XML before importing, and replace the start and end tag from JiveMessenger to Wildfire. I consider this not a bug, but a real annoyance: the User import/export plugin could easily support old formats, to make their own users to upgrade their server...
This time, it took me less than 10 minutes. Yes, it's that easy. And I'll bet that you could even do it in less time, but I have to patch the wildfire startup file to my personal taste.
The patch to the wildfire script allows me to run Wildfire in the foregroun, under daemontools. Its simple: copy the start section of the shell script and rename it as foreground and then remove the nohup at the beginning of the following line and the & at the end. There, you can now supervise your Wildfire installation.
You can fetch (.tar.gz, .tgz, .zip) my patch to the wildfire startup script, and my run files that I use for my setup. Please don't use them without reading. I have strange layouts in my servers.
Anyway, simplicidade.org domain is now up and running again, with a full XMPP compliant server.
Note: Robin Bowes commented that he was having difficulties with the tarball... I tried it just now and tar zxf wildfire_supervised.tar.gz worked first time. Anybody else had problems?
Note 2: Pedro also has problems with the file. I don't know what's going on. I can do a tar zxf both on the server side and on the client side after downloading with Safari. In any case I placed two more options for download, a .tgz and .zip. Maybe that helps.
Technorati Tags: ejabberd, jabber, jivesoftware, wildfire, xmpp
From a Ars.Technica article:
Google Talk may have a spartan user interface and a small user base, but sometime in the near future, ICQ and AIM will be able to communicate directly with GTalk servers as well. That means that all the XMPP networks can mingle with the millions of ICQ/AIM users, and the federated love-in I was hoping for is one step closer to reality.
Oahh there... Just because AIM/AOL will open up federation to Google Talk, that federation is not transitive. I cannot route my XMPP traffic from my personal domain via Google Talk network.
AIM already has federation deals that you can sign with them, but they are pretty expensive.
Having said that, I would love to see the XMPP world gaining 50+ million new JIDs...
Technorati Tags: aol, google, aim, googletalk
Ok, I was able to subscribe several buddies @gmail.com just now, and talk to them, all of this from my @sapo.pt account.
Server-to-server is now open with GTalk.
Let the games begin.
(kudos to Rusty Shackleford who pointed this to me)
update: it seems that ralphm beat me to it. Behold the power of mimir (speaking of which, I got to resubscribe to it...).
update 2: Official announcement via GoogleTalk blog.
Technorati Tags: google, googletalk, jabber, xmpp
The server-to-server port of the google.com domain is now open via XMPP. You should be able to talk to all your friends working at Google.
Gmail server is still not open. If we assume that they are using the same code at google.com and gmail.com, it should be safe to assume that this is a external interop test, to prepare the full opening.
Cool.
Update: thanks to Rusty Shackleford (see his comment below) for pointing out that GTalk is now open.
Technorati Tags: google, googletalk, xmpp
For those of us working with XMPP, this is probably the best christmas present we could have.
Yesterday, the Jabber Software Foundation, announced two new JEPs (similar to the RFCs of IETF, they define extensions to the basic XMPP RFCs), Jingle Signalling and Jingle Audio, to specify a standard way for XMPP clients to negotiate Voice over IP sessions.
In the wings of that announcement, Google released libjingle (also look at the SourceForge Project page for libjingle), a C++ library to implement the Jingle spec.
I'm still looking over licensing details and technical details, but the future for XMPP-based VoIP seems a lot brighter today.
BTW, my employer, is happy and proud to be part of the list of companies that pledged support for this standard. :)
Update: I'll be collecting some links to post around the comunity
Update 2: Aparently the libjingle also includes a relay server and a STUN server. From the libjingle readme file (via Celso):
Relay Server
Libjingle will also build a relay server that may be used to relay traffic when a direct peer-to-peer connection could not be established. The relay server will build in talk/p2p/base/relayserver and will listen on UDP ports 5000 and 5001. See the Libjingle Developer Guide at http://code.google.com/apis/talk/index.html for information about configuring a client to use this relay server.
STUN Server
Lastly, Libjingle builds a STUN server which implements the STUN protocol for Simple Traversal of UDP over NAT. The STUN server is built as talk/p2p/base/stunserver and listens on UDP port 7000. See the Libjingle Developer Guide at http://code.google.com/apis/talk/index.html for information about configuring a client to use this STUN server.
Technorati Tags: google, googletalk, jabber, jingle, libjingle, network, xmpp
From the Gaim site:
I (Sean) have been hired by Google, moved to Seattle, and have been working on the Google Talk team for about a month and a half. The goal of Google Talk is to make real-time communication as open as possible, and in that regard, I've been working to offer all of Google Talk's features into other clients. Currently, I'm working on making it as easy as possible for other clients to use Google Talk's voice features. You can expect Gaim and other clients to be interoperable with Google Talk's voice features in the near future.
Good, maybe they could start by documenting the protocol and let us all help out.
I've got a couple of GMail invites.
If you want to get an email account or try the Talk thing, get a Jabber client, create an account on a free server, and send me a Jabber message with your email address.
My Jabber ID is melo@simplicidade.org.
I'll send you your invitation.
I had though about doing this some weeks ago when I got them, but I forgot. Thank's to stpeter for the reminder.
Guys, yes, I agree that the Google Talk client is too simple:
That's great!
That only means that we can build our own clients, with those features, and compete. Imagine that, real competition in the IM world. The best client wins. I'm not letting go of my Psi, I can tell you that.
And Yahoo, AOL, Apple and Google will always be able to innovate and integrate services. There is no big risk to them, to provide extra services to their customers, using their clients.
The difference is that, like SMTP for mail, they will all be using a standard and open protocol.
That's the real issue about Google Talk: they are using something open and therefore, you can compete with them on the client side.
If you don't understand what I'm saying, try this:
The only thing for this to become a reality is server-to-server interop. :)
There are a lot of servers that you can install and join the XMPP world. There is an up-to-date server list at the Jabber Foundation website.
My current recomendation goes to Jive Messenger. Very easy to setup and to manage.
Also, for a turn-key solution for your company, your should look at Jabber Now.
This is the current list of things to discover about the usage of XMPP in Google Talk.
Updated as I discover all of this.