Psi on Notes

Google Talk client is too simple = Great!!!

melo@simplicidade.org (Pedro Melo) — Wed, 24 Aug 2005 19:57:33 +0000

Guys, yes, I agree that the Google Talk client is too simple:

no offline messages;
no “spiffy” flash stuff and emoticons with gazilion skins;
no file transfer;
no multi-user chat;
etc, etc, etc.

That’s great!

That only means that we can build our own clients, with those features, and compete. Imagine that, real competition in the IM world. The best client wins. I’m not letting go of my Psi, I can tell you that.

And Yahoo, AOL, Apple and Google will always be able to innovate and integrate services. There is no big risk to them, to provide extra services to their customers, using their clients.

The difference is that, like SMTP for mail, they will all be using a standard and open protocol.

That’s the real issue about Google Talk: they are using something open and therefore, you can compete with them on the client side.

If you don’t understand what I’m saying, try this:

The only thing for this to become a reality is server-to-server interop. :)

Psi nightly builds

melo@simplicidade.org (Pedro Melo) — Fri, 15 Apr 2005 11:38:14 +0000

So I’m a lazy bastard, so far nothing new. But I also want to look at the latest Psi, ze mozt powerfull Jabber client out there, releases for Mac, and lo and behold they have nightly builds…

This is the script I ended up with to update my Psi every day:

melo@RoadRunner(~/Desktop/psi-night-build)$ cat update_psi.sh
#!/bin/sh
#
# Updates Psi with latest nightly build
#

today=`date +%Y%m%d`

if [ -d "psi-mac-$today" ] ; then
  echo "You already have build $today, restart your Psi to use it, maybe?"
  exit 1
fi

curl -O http://spike.ulyssis.org/psi/mac/psi-mac-$today.zip

unzip psi-mac-$today.zip
rm psi-mac-$today.zip
rsync -a into_frameworks/ psi-mac-$today/Psi.app/Contents/Frameworks
rm -f psi-latest
ln -s psi-mac-$today psi-latest
growlnotify -n "Psi Nightly build updater" -s -a Psi 'Psi is updated!' <<EOF
You Psi was updated with the latest nightly build, $today. Please Quit and restart you Psi.
EOF

Make sure you chmod 755 update_psi.sh.

Now you just need to restart you Psi every morning after the nice growl notification appears. I keep ~Desktop/psi-night-build/psi-latest/Psi.app on my Dock.

Couple of notes: on the directory where you run update_psi.sh, you must have a directory into_frameworks. Inside you need to put all the Frameworks required to run Psi. Right now, you need to have the libqt-mt.3.dylib (found on the same directory where you can find Psi nightly builds), and the Growl framework.

I used the copy of the Growl framework that comes with Ecto. Use locate Growl.framework to find a copy on your system, and copy it into into_frameworks.

My into_frameworks looks like this:

melo@RoadRunner(~/Desktop/psi-night-build)$ ls -la into_frameworks/
total 18376
drwxrwxr-x  4 melo  melo      136 15 Apr 11:01 .
drwxrwxr-x  9 melo  melo      306 15 Apr 11:29 ..
drwxr-xr-x  7 melo  melo      238  4 Apr 04:17 Growl.framework
-rw-r--r--  1 melo  melo  9404804 16 Aug  2004 libqt-mt.3.dylib

The main directory where I keep update_psi.sh looks like this:

melo@RoadRunner(~/Desktop/psi-night-build)$ ls -la
total 128
drwxrwxr-x   9 melo  melo    306 15 Apr 11:29 .
drwx------  23 melo  melo    782 15 Apr 11:21 ..
drwxrwxr-x   4 melo  melo    136 15 Apr 11:01 into_frameworks
lrwxrwxr-x   1 melo  melo     16 15 Apr 11:12 psi-latest -> psi-mac-20050415
drwxr-xr-x   5 melo  melo    170 15 Apr 02:25 psi-mac-20050415
-rwxr-xr-x   1 melo  melo    598 15 Apr 11:27 update_psi.sh

Although the Growl framework is required in latest builds, I cannot seem to get it working. If you know how, xmpp me at melo funnychar simplicidade.org.

Update: Growl works very well if you use the Remko versions. See his Psi page.

Update 2: small bug in the if [ -d ... in the script, change “ to “”. Thanks go to rebarbado.

XMPP, Jabber and Growl: notification systems

melo@simplicidade.org (Pedro Melo) — Thu, 02 Dec 2004 13:30:44 +0000

Updated: fixed all the links, sorry, I’m a markdown beginner. Thanks Ralph.

Rui did some work on growl remote notifications, and then he made a question: why don’t we make this a standard? In doing that, he also talks about Jabber and the failure of that project. I don’t agree with the last part at all, and I’ll try to answer his question.

First, let’s talk about Growl. I really really like Growl, I think it is the second best thing that has happened to the Mac this year (the first being Quicksilver). I like the fact that with it I can have a single app and a single prefpane that controls all my notifications. I even changed IM clients (from Proteus to Adium) because I could not find Growl support in Proteus, and Adium 0.7 had it.

But Growl has some problems. First, I don’t have a history of notifications. Sometimes I’m away from my TiBook, and I lose notifications. I don’t have a place where I can see the last 20 minutes of notifications.

Second, when I’m offline, I cannot receive notifications, and while the servers could keep on trying to send them, it wouldn’t work very well. The moment you get online, you’re bombarded with notifications, so you’ll probably just skip most of them. If you want to see what I’m saying, download Rawr-endezvous, connect your Mac to a local network with lots of other Macs, and fire it up. Expect to be looking at several screens worth of notifications.

Third, you can specify a title and a text for each notification, but you cannot follow it through to the tool/site/server where you can fix the problem. Growl really needs support for a URL, a clickable one. The lack of a “tell me more”-action limits the usefulness.

Fourth, I have 200+ systems, in several places, behind several types of firewalls. My own laptop is behind another corporate firewall, and I don’t expect the local firewall admins to open up yet another port for notifications. Also, I don’t think it is reasonable to have systems on the other side of the planet sending me notifications directly to my laptop.

Growl is the last mile of notification systems. It’s a personal notification system, not a network notification system. Does it need to receive events from the network? Sure! Should we create another protocol for that? Maybe not.

A network notification system needs a lot more than displaying the notifications. It needs aggregation of events, it needs to know that the user is online or offline, to route the notifications to the most available person to handle it. And that brings us to IM systems.

If you use an IM system for transport of notifications, you get a lot of those things for free. When I say IM system, I don’t limit myself to XMPP/Jabber. You can use SIMPLE if you want. I really don’t care. I use XMPP/Jabber because that’s what I know. I’ll use XMPP in the rest of this article because of that. Fell free to s/XMPP/IM system/g.

With XMPP you get a lot of things out-of-the-box: * You get an extensible protocol; * You get all the authentication stuff (with TLS support); * you get a bi-directional protocol, so you can ask the source of the notification “Tell me more”; * you get a global scalable reliable protocol; * you get powerful publish/subscribe primitives.

Let’s look at each of these in turn. XMPP is an extensible protocol and that allows you to define new packets (stanzas in XMPP lingo) to transport all the information and semantic hints you want.

You can ask people who know me, I’m not a big fan of XML. I think it is over-hyped and over-used. It seems that, like IBM mainframes in the 60’ and Cisco gear in the 90’, you won’t be fired if you use them. XML is also lousy for transport of information. Network protocols should be efficient and XML is not. An XML parser is more complex and heavy than a XDR or ASN.1 parser, for sure. Also the amount of useless information in XML is very high. Huffman must be turning in it’s grave.

I don’t see XML as the reason for XMPP extensibility, I really don’t, but XML is a very high-level method of encoding information, and for now it helps to use it as the base of XMPP. Eventually, you could put a XML dictionary-based compression scheme between XMPP clients and servers, and still use the same protocol. Only compressed in transit. Gzip is also an option, and it’s even mentioned in XMPP RFC (this is from memory, I need to check the negotiation phase).

Before people start telling me something like “you stupid person, my p4 x ghz machine does not have performance problems with XML”, or “you moron, my 2mbit ADSL is mostly empty, so if I need 400 bytes to send a simple hello, that’s not a problem”, I have two things for you to think about: * always-on mobile networks will charge you for bytes transfered, at least in the mass-market plans; * it’s a waste of resources having 20000 clients connected to my server and having 20000 XML parsers: machines don’t need human- readable protocols.

Back to our list, the second point is TLS support. It’s mandatory in XMPP. It’s a standard, it works and it’s well understood.

The third point is bi-directional protocol. Having a bi-directional protocol allows you to use your notification system to query your systems. It allows your notifications to be short and to the point, giving you the possibility to ask for more information, so that you can correctly evaluate the problem.

The fourth point, a scalable network, is very important if you have more than 50 or so servers. You need aggregation points for notifications, and you need server-to-server communications. You also need reliability, you need the assurance that if an application sends a notification to the local notification center, the message wont be lost (insert regular acts of god disclaimer here). You need store-and-forward semantics, or end-to-end acknowledge of notification. Maybe a mix of both.

Finally, you probably work in a team, with a on-duty staff, and with escalation procedures for notifications. If a disk is 100% full, I don’t care if you’re offline, someone must be notified, right? Here we get into pubsub systems, probably the best way to deal with the problem of one to many communications right now. You can subscribe to applications, servers, clusters, any node or object in your infrastructure. It’s your decision, it’s under your control.

Pubsub is available in XMPP in two forms: the lazy one, and the standard one. The lazy one is multi-user chat rooms. You can use a multi-user chat room as a basic quick and dirty pubsub. Just create a chat room for each topic. Whoever wants to be notified, just joins the chat room. It’s a one-to-many system, and it works. Imagine this: all your mail servers connected to the same chat room, and having one of them say “Hey people, this dude at 127.0.0.1 is spamming me hard. block the sucker”, and having all the other machines in the cluster do it. You can even go into the chat room to monitor those events, or even to say “guys, unblock 127.0.0.1, please”. Yes, notifications can be sent between applications.

The standard for pubsub is JEP-0060, and you have at least two implementations today: ralph’s Idavoll and ejabberd (which also seems a very nice, mostly-XMPP compliant server. If only virtual hosts where supported…)

So with all of this, using an IM protocol for transport of notifications has a lot going for it. XMPP is by no means perfect. I’m still not sure if XMPP can really assure delivery of messages between servers. I don’t think you have the same store-and-forward with acknowledges you have in SMTP. And I really think that a low-bandwidth version of the Jabber network protocol is needed. But it’s a start.

So answering Rui’s question: why not make Growl a standard? Well, by all means, do it. But as a last mile solution for the notification problem, not for the network part of the problem. The network part is complex and we already have protocols that do it very well, we just need to glue them together.

How do we do that? Well, I have an idea, but I’ll need to talk about something else first, to clarify something about XMPP/Jabber that Rui also mentions (yes, the “jabber is dead part”…).

Saying that XMPP is an IM protocol, is like saying that SMTP is for sending text messages. Yes, you can do it, and it’s probably the most common use. But XMPP is so much more if you look into it. It has a powerful resource discovery protocol (See JEP-0030) and at least one decent client implementation, Psi, very decent publish and subscribe capabilities, and it also does IM :).

Regarding the lack of clients, my answer is “it depends”. You can have great clients today if you want the IM part only. Adium works very well, Fire also works OK. If you really need a XMPP client (and that’s an entirely different ball game), then no, there aren’t any good clients out there. Again, Psi is probably the best one.

The way I see it, XMPP is much more powerful than traditional IM systems, and most client implementations stop after having the IM part done. The rest of the XMPP doesn’t seem to have reached critical mass. In XMPP, you can use pubsub to publish the current music you are listening to, and I can subscribe to that topic to know that (and then probably use Growl to tell me whenever that changes). But there is a lot of framework missing that needs to be in place to make it work: how does my XMPP client talk to my music player? In a Apple environment this is a little bit easier, due to the iTunes domination, but these are the problems that face XMPP extensions adoption for now.

Now back to Rui’s answer: how do we do it? Well, you could build a XMPP client not for IM, but for notifications only. Your own personal notification avatar. Julian mentioned this recently. You can even built it into Growl itself.

This would allow you to use Growl to subscribe to topics. You still need to solve Growl shortcomings (at least the lack of URL support), but the rest is pretty much already in place.

My own jab at the problem, Ambrosio, does some of it, but I’m still finishing it’s infrastructure to be able to do what I’ve been talking about. We’ll see what happens in the next weeks.