Random Etc. Notes to self. Work, play, and the rest.

Firehose

Update (June 20th): the 'Firehose' app described in this post relied on Twitter's public timeline IM support, which has been down for several days in a row. Firehose will return when Twitter gets their stuff back online.

firehose.stamen.com

About a week ago I made a proof of concept Flash site that I'm calling Firehose. It shows every post from Twitter's public timeline as fast as it can, in a BigSpy style.

I wasn't sure whether to make it public any time soon, but given how interesting people are finding sites like Twistori the time seems right. Blaine mentioned it on the Gillmor Gang podcast yesterday, and it hasn't gone down since so I assume it will cope with more viewers!

I'm using the XIFF actionscript 3 library to speak to a server running OpenFire, and that server has a bot that broadcasts items from Twitter's public timeline as XMPP (Jabber) instant messages to everyone viewing the Flash site. The main reason it needs its own server is because for socket communication Flash is only allowed to connect to the domain it was served from, and therefore it can't connect to Google Talk or AIM from any server I can host it on.

Anyway, you can probably tell that I'm more interested in the technology than whether it's useful or not. It really is just a proof of concept. It's a bit silly because it's too fast to be readable. But it works! (Except when it doesn't. Let me know in the comments if it doesn't work for you?)

Silly or not, Firehose does expose one tiny piece of functionality right now. There's no filtering yet, but it currently highlights "twitter" by default, and you can see what it looks like with other words highlighted by using the URL #fragment, like this, or this, or this.

Clearly there are several next steps I could take with this. The same setup could also be used to subscribe to a particular set of users, or tracked keywords, or (with a little more server-side work) to geocode the tweets and plot them on a map, TwitterVision style. We're just getting started with this, and Twitter is the only public jabber bot I know of with this kind of volume of output (this BBC one looks interesting too though).

NB:– Twitter is a former client of my employer Stamen but we're not working with them officially at the moment. This is the same Jabber PubSub feed they made public a few weeks back, that people are mainly using to power Twitter search engines.


6 Comments

Tom, I’m getting a ‘Searching for the firehose … Not Authorized’ message (top right of screen).

Posted by David Smith on 17 May 2008 @ 12am

Thanks David. I’ll drop you an email to find out whether it’s something to do with my code, or your configuration, or a temporary glitch.

Posted by TomC on 17 May 2008 @ 3pm

“Service unavailable” for me, Tom.

Posted by Ben on 20 May 2008 @ 6am

Sadly the Twitter bot is up and down this morning, so I’ve taken Firehose offline for now.

Posted by TomC on 20 May 2008 @ 11am

Ben - for posterity, I think I figured out some of the issues I was having with the Flash library - in the latest version of the plugin you have to expressly ask the server for permission to make socket connections. That seems to clear up some issues with 9.0.124.0 - you can see what version you’re running at http://www.adobe.com/products/flash/about

Posted by TomC on 22 June 2008 @ 11am

[...] Firehose http://www.tom-carden.co.uk/2008/05/16/firehose/ - bookmarked by 1 members originally found by LadyVixUk on 2008-07-21 Mobile Jabber clients for Symbian/S60 phones? http://huima.wordpress.com/?p=151 - bookmarked by 2 members originally found by nsemenza on 2008-07-18 [...]

Posted by Bookmarks about Openfire on 15 August 2008 @ 9am