Firehose

16th May 2008 @ 10:55 pm
nowhere
Update (June 20th): the 'Firehose' app described in this post relied on Twitter's public timeline IM support, which has been down for several days in a row. Firehose will return when Twitter gets their stuff back online.

firehose.stamen.com

About a week ago I made a proof of concept Flash site that I'm calling Firehose. It shows every post from Twitter's public timeline as fast as it can, in a BigSpy style.

I wasn't sure whether to make it public any time soon, but given how interesting people are finding sites like Twistori the time seems right. Blaine mentioned it on the Gillmor Gang podcast yesterday, and it hasn't gone down since so I assume it will cope with more viewers!

I'm using the XIFF actionscript 3 library to speak to a server running OpenFire, and that server has a bot that broadcasts items from Twitter's public timeline as XMPP (Jabber) instant messages to everyone viewing the Flash site. The main reason it needs its own server is because for socket communication Flash is only allowed to connect to the domain it was served from, and therefore it can't connect to Google Talk or AIM from any server I can host it on.

Anyway, you can probably tell that I'm more interested in the technology than whether it's useful or not. It really is just a proof of concept. It's a bit silly because it's too fast to be readable. But it works! (Except when it doesn't. Let me know in the comments if it doesn't work for you?)

Silly or not, Firehose does expose one tiny piece of functionality right now. There's no filtering yet, but it currently highlights "twitter" by default, and you can see what it looks like with other words highlighted by using the URL #fragment, like this, or this, or this.

Clearly there are several next steps I could take with this. The same setup could also be used to subscribe to a particular set of users, or tracked keywords, or (with a little more server-side work) to geocode the tweets and plot them on a map, TwitterVision style. We're just getting started with this, and Twitter is the only public jabber bot I know of with this kind of volume of output (this BBC one looks interesting too though).

NB:– Twitter is a former client of my employer Stamen but we're not working with them officially at the moment. This is the same Jabber PubSub feed they made public a few weeks back, that people are mainly using to power Twitter search engines.