Updated: fixed all the links, sorry, I'm a markdown beginner. Thanks Ralph.
Rui did some work on
growl remote notifications, and then he made a question: why don't
we make this a standard?
In doing that, he also talks about Jabber and the failure of that
project. I don't agree with the last part at all, and I'll try to answer
his question.
First, let's talk about Growl. I really really like Growl, I think it is
the second best thing that has happened to the Mac this year (the first
being Quicksilver). I like the fact that with it I can have a single app
and a single prefpane that controls all my notifications. I even changed
IM clients (from Proteus to Adium) because I could not find Growl
support in Proteus, and Adium 0.7 had it.
But Growl has some problems. First, I don't have a history of
notifications. Sometimes I'm away from my TiBook, and I lose
notifications. I don't have a place where I can see the last 20 minutes
of notifications.
Second, when I'm offline, I cannot receive notifications, and while the
servers could keep on trying to send them, it wouldn't work very well.
The moment you get online, you're bombarded with notifications, so
you'll probably just skip most of them. If you want to see what I'm
saying, download Rawr-endezvous,
connect your Mac to a local network with lots of other Macs,
and fire it up. Expect to be looking at several screens worth of
notifications.
Third, you can specify a title and a text for each notification, but you
cannot follow it through to the tool/site/server where you can fix the
problem. Growl really needs support for a URL, a clickable one. The lack
of a "tell me more"-action limits the usefulness.
Fourth, I have 200+ systems, in several places, behind several types
of firewalls. My own laptop is behind another corporate firewall, and
I don't expect the local firewall admins to open up yet another port
for notifications. Also, I don't think it is reasonable to have
systems on the other side of the planet sending me notifications
directly to my laptop.
Growl is the last mile of notification systems. It's a personal
notification system, not a network notification system. Does it need to
receive events from the network? Sure! Should we create another protocol
for that? Maybe not.
A network notification system needs a lot more than displaying the
notifications. It needs aggregation of events, it needs to know that the
user is online or offline, to route the notifications to the most
available person to handle it. And that brings us to IM systems.
If you use an IM system for transport of notifications, you get a lot of
those things for free. When I say IM system, I don't limit myself to
XMPP/Jabber. You can use SIMPLE if you want. I really don't care. I use
XMPP/Jabber because that's what I know. I'll use XMPP in the rest of
this article because of that. Fell free to s/XMPP/IM system/g.
With XMPP you get a lot of things out-of-the-box:
* You get an extensible protocol;
* You get all the authentication stuff (with TLS support);
* you get a bi-directional protocol, so you can ask the source of the
notification "Tell me more";
* you get a global scalable reliable protocol;
* you get powerful publish/subscribe primitives.
Let's look at each of these in turn. XMPP is an extensible protocol and
that allows you to define new packets (stanzas in XMPP lingo) to
transport all the information and semantic hints you want.
You can ask people who know me, I'm not a big fan of XML. I think it is
over-hyped and over-used. It seems that, like IBM mainframes in the 60'
and Cisco gear in the 90', you won't be fired if you use them. XML is
also lousy for transport of information. Network protocols should be
efficient and XML is not. An XML parser is more complex and heavy than a
XDR or ASN.1 parser, for sure. Also the amount of useless information in
XML is very high. Huffman must be turning in it's grave.
I don't see XML as the reason for XMPP extensibility, I really don't,
but XML is a very high-level method of encoding information, and for now
it helps to use it as the base of XMPP. Eventually, you could put a XML
dictionary-based compression scheme between XMPP clients and servers,
and still use the same protocol. Only compressed in transit. Gzip is
also an option, and it's even mentioned in XMPP RFC (this is from
memory, I need to check the negotiation phase).
Before people start telling me something like "you stupid person, my
p4 x ghz machine does not have performance problems with XML", or "you
moron, my 2mbit ADSL is mostly empty, so if I need 400 bytes to send a
simple hello, that's not a problem", I have two things for you to
think about:
* always-on mobile networks will charge you for bytes transfered, at
least in the mass-market plans;
* it's a waste of resources having 20000 clients connected to my
server and having 20000 XML parsers: machines don't need human-
readable protocols.
Back to our list, the second point is TLS support. It's mandatory in
XMPP. It's a standard, it works and it's well understood.
The third point is bi-directional protocol. Having a bi-directional
protocol allows you to use your notification system to query your
systems. It allows your notifications to be short and to the point,
giving you the possibility to ask for more information, so that you can
correctly evaluate the problem.
The fourth point, a scalable network, is very important if you have more
than 50 or so servers. You need aggregation points for notifications,
and you need server-to-server communications. You also need reliability,
you need the assurance that if an application sends a notification to
the local notification center, the message wont be lost (insert regular
acts of god disclaimer here). You need store-and-forward semantics, or
end-to-end acknowledge of notification. Maybe a mix of both.
Finally, you probably work in a team, with a on-duty staff, and with
escalation procedures for notifications. If a disk is 100% full, I don't
care if you're offline, someone must be notified, right? Here we get
into pubsub systems, probably the best way to deal with the problem of
one to many communications right now. You can subscribe to applications,
servers, clusters, any node or object in your infrastructure. It's your
decision, it's under your control.
Pubsub is available in XMPP in two forms: the lazy one, and the standard
one. The lazy one is multi-user chat rooms. You can use a multi-user
chat room as a basic quick and dirty pubsub. Just create a chat room for
each topic. Whoever wants to be notified, just joins the chat room. It's
a one-to-many system, and it works. Imagine this: all your mail servers
connected to the same chat room, and having one of them say "Hey people,
this dude at 127.0.0.1 is spamming me hard. block the sucker", and
having all the other machines in the cluster do it. You can even go into
the chat room to monitor those events, or even to say "guys, unblock
127.0.0.1, please". Yes, notifications can be sent between applications.
The standard for pubsub is JEP-0060, and you have at least two
implementations today: ralph's Idavoll and ejabberd (which
also seems a very nice, mostly-XMPP compliant server. If only virtual
hosts where supported...)
So with all of this, using an IM protocol for transport of notifications
has a lot going for it. XMPP is by no means perfect. I'm still not sure
if XMPP can really assure delivery of messages between servers. I don't
think you have the same store-and-forward with acknowledges you have in
SMTP. And I really think that a low-bandwidth version of the Jabber
network protocol is needed. But it's a start.
So answering Rui's question: why not make Growl a standard? Well, by all
means, do it. But as a last mile solution for the notification problem,
not for the network part of the problem. The network part is complex and
we already have protocols that do it very well, we just need to glue
them together.
How do we do that? Well, I have an idea, but I'll need to talk about
something else first, to clarify something about XMPP/Jabber that Rui
also mentions (yes, the "jabber is dead part"...).
Saying that XMPP is an IM protocol, is like saying that SMTP is for
sending text messages. Yes, you can do it, and it's probably the most
common use. But XMPP is so much more if you look into it. It has a
powerful resource discovery protocol (See JEP-0030) and at least
one decent client implementation, Psi, very decent publish and
subscribe capabilities, and it also does IM :).
Regarding the lack of clients, my answer is "it depends". You can have
great clients today if you want the IM part only. Adium works very well,
Fire also works OK. If you really need a XMPP client (and that's an
entirely different ball game), then no, there aren't any good clients
out there. Again, Psi is probably the best one.
The way I see it, XMPP is much more powerful than traditional IM
systems, and most client implementations stop after having the IM part
done. The rest of the XMPP doesn't seem to have reached critical mass.
In XMPP, you can use pubsub to publish the current music you are
listening to, and I can subscribe to that topic to know that (and then
probably use Growl to tell me whenever that changes). But there is a lot
of framework missing that needs to be in place to make it work: how does
my XMPP client talk to my music player? In a Apple environment this is a
little bit easier, due to the iTunes domination, but these are the
problems that face XMPP extensions adoption for now.
Now back to Rui's answer: how do we do it? Well, you could build a XMPP
client not for IM, but for notifications only. Your own personal
notification avatar. Julian mentioned this recently. You can even
built it into Growl itself.
This would allow you to use Growl to subscribe to topics. You still need
to solve Growl shortcomings (at least the lack of URL support), but the
rest is pretty much already in place.
My own jab at the problem, Ambrosio, does some of it, but I'm
still finishing it's infrastructure to be able to do what I've been
talking about. We'll see what happens in the next weeks.
