Random Etc. Notes to self. Work, play, and the rest.

Posts Tagged ‘Processing’

Greetings Tube Fans

Thanks to some curious emails and a couple of dormant Google Alerts, it's come to my attention that the Travel Time Tube Map I made a few years ago has had a sudden resurgence of internet fame. My original blog post informs me that it's over 5 years old. Wow!

I'm not sure who rediscovered it first, but thanks to everyone who's linked to it so far including Fast Co. Design, Creativity Online, Wired UK, PSFK, Roomthily, Inteloquent, OpenStreetMap and numerous Twitter and Facebook users.

The map has been picked up by a few books and exhibitions over the years, including the wonderful Form + Code by Casey Reas and Chandler McWilliams. If you're interested in how this kind of work gets made then the book is essential.

If you're interested in a more thorough theoretical exploration of isochrones I can recommend Nicholas Street's Time Contours paper on the subject. If you find yourself yearning for an even deeper treatment of transit data, look around for people like Mike Frumin who take research far more seriously than I do!

If you want to play around with this code for yourself, it should be relatively easy to fix up for current versions of Processing (probably just the fonts will need updating, please leave a comment if there's anything else) and you can get the data here.

I've had a few requests to update the map with current data, including the East London Line and Heathrow Terminal 5, as well as suggestions to include the overground in south London and elsewhere. Sadly I haven't found a coherent and consistent data source that I could drop-in as a replacement for my hand-edited original. The official Transport for London data sources on data.gov.uk look promising, and I've had a couple of under-the-table offers from people with access to time-table data, but these all require more time and effort than I have for the map at the moment. In future I'd like to move the map to a more 201x format like Canvas or SVG, perhaps porting to Processing JS. Perhaps an app? One day...

ArtNano (notes on Processing for design elements)

As I mentioned in my previous post, we were working with Geraldine Sarmiento on the ArtNano site, and she came up with the pixellised imagery you see throughout.

Towards the end of the project, I needed a few more of the pixelly images at short notice, to illustrate the essays and about pages on the site. Rather than bother Geraldine, I reached for Processing to see if I could match the look of the homepage imagery that she had created.

I came up with an applet that used a bit of blurring, a bit of distance fall-off, and a bit of perlin noise to create the effect that we were looking for (decorative, obscured, but related to the overall site). Here's an example of the imagery created from a picture of Scott Snibbe's Three Drops that we used on the page for Jennifer Frazier's essay:

Three Drops pixels applet

You can see the full applet and source code here, and I think it's a good example of how a generative solution to design elements can keep a project flexible right up until launch (and beyond).

Processing Blogs: The Internet Hates Me

Sorry for the recent outage on Processing Blogs. For some reason all the wordpress plugins deactivated themselves. I hate the internet, and the internet hates me.

Processing Blogs Fixed II: Electric Boogaloo

Let's try this again. If you're reading this on Processing Blogs or via its feed, then everything should be working.

Instead of Feedwordpress and Planet Planet put together with sticky tape and glue, I'm using the better documented Planet Venus and WP-Venus so our archives are maintained.

If you ever need to run a site similar to Processing Blogs and your web host can run python then I definitely recommend Planet or Planet Venus as the solution. WP-Venus complained a little bit when I converted from Feedwordpress, but it looks good so far: hopefully the archives will be worth the effort.

Processing Blogs Fixed

If you're reading this on Processing Blogs, or via its feed, then everything should be fixed. For some reason, Feedwordpress just stopped working and wouldn't re-subscribe to lots of the feeds that were previously fine.

Feeds are now being grabbed robustly by Planet and merely massaged by Feedwordpress. Here's the current list:

Please do let me know if I'm syndicating too much or too little, and especially if I'm missing people or if I'm syndicating your blog and you don't want me to. Otherwise, the site should run itself for a while.

Design.Yahoo: Web 2.0 meets War Games?

Whilst I wrestle with my reaction to the reception of Twitter Blocks, it's interesting to look at what other people in information visualisation are working on.

Yahoo Design

Yahoo's new design research outfit, apparently also known as yhaus, have just put up a site outlining their work so far. The first thing I noticed is that they've snagged an amazing subdomain: design.yahoo.com. The second thing I noticed is that compared to other teams I'm aware of in the field (including Stamen) they have a pretty good gender balance (a thorny issue but I'm noting it anyway). The third thing I'll note is the guest appearance of non-Yahoo work in the portraits of the team: I see Torrent Raiders, Fidg't... what else?

Sadly, all but one of the demos there so far is a big Quicktime movie. I know that with millions of users Yahoo has to be a stickler for browser support and compatibility, but I hope they get a chance to take this work live on the web as well as demo it in movie form. There's some solid realtime Flash and Processing work hiding in there, and people (OK, I) want to see it in its interactive entirety.

There's clearly some healthy collaboration and influence going on there (much as in our work, e.g. Ben Fry's zipdecode looms large over the interactive version of our Trulia search animation). Yahoo's Aaron Koblin is best known for his Flight Patterns piece, and this visualisation by Michael Chang of Yahoo trip planner data is very similar:

Yahoo Travel

Likewise, Aaron's work on traffic patterns bears a close resemblance to Flight Patterns:

Yahoo Traffic

I don't want to pick on them too much, because it's really beautiful work I admire a great deal (and it might seem like sour grapes), but both the pieces I've highlighted do suffer from something we've tried to avoid at Stamen: animated information graphics on top of black backgrounds and vector maps can easily look like screenshots from a modern-day War Games:

War Games: Missile Warning

I'm glad other people as well as us are experimenting in public, and I'm glad sites like Infosthetics and Visual Complexity are cataloguing our efforts. We need our own visual language around this kind of visualisation that doesn't resonate with the imagery of war.

Apple's iphone has made a strong impression with slick transitions in its interface design, but the maps application still borrows from Google's pseudo-shadows and static pins. The playful interfaces of the Nintendo's Wii games certainly offer a different path, but the rest of the games (and movie) industry's cinematic-realism aesthetic exerts a strong influence over our generation of designers and it isn't going to meet these goals any time soon.

It's a fairly regular topic of conversation at Stamen: how can you make a visualisation of e.g. 911 calls actually look like emergencies, and not birthday parties or toilet flushes, without freaking people out and without making it mundane? Is it possible to use great circles to connect air travel destinations without it looking like missiles? Can you animate growing and shrinking red and yellow circles on an aerial map without it looking like Gulf War I weapons company propaganda?

It's a bloggy weekend here at Random Etc, if you're reading along I'm definitely interested to hear your thoughts.

Criticism for Twitter Blocks

Twitter Blocks Launched!

We finally kicked Twitter Blocks out of the door yesterday. It's been in development for about a month, mainly by Ryan Alexander and myself (but all Ryan whilst I was working on Oakland Crimespotting, he's a star). It's the first time we've done 3D things using Flash and it's amazing what's being done with Papervision3D at the moment. On the other hand, having 3D shoe-horned into the 2D Flash engine means the learning curve is a lot steeper than the native-3D Processing/OpenGL worlds Ryan and I are used to.

Whenever Stamen launches a new thing, my immediate reaction is a sigh of relief followed by a slightly obsessive-compulsive trawl of what people are saying about it online. (I use a combination of alerts from Technorati, Google Blog Seach and Bloglines to keep track).

The interesting thing about working with companies like Digg and Twitter is that your work inherits all the criticisms and detractors of those sites as well. Digg's users are clamouring for a picture section, so when we launched Digg Arc many of the responses ignored the piece entirely and chastised Digg for paying attention to visualisation and not to the main site. The same argument is already being used against Twitter Blocks, even though the amount of time Twitter's developers put into it was tiny compared to the amount of time they're putting into stability and new features.

Don't get me wrong: some of the early feedback we're getting is very positive, the team at Twitter have been very receptive and we're proud of our work. This much is good. Some of the feedback we're getting points out that the work isn't immediately understandable (I agree, and maybe we could do some more explaining, but I think we're OK for now).

However, there is also a strong and steady flow of negative comments that I've gathered here so I can think about them all in one place.

“Pretty visualization but I doubt its practicality.” PoppuPot

“Twitter Blocks is the kind of thing that demos well at conferences. Not too useful in real life.” Dave Winer

“exploring myneighbourhood : fun 3D view but so what? not sure i will do that everyday.” jean-michel gobet

“Well its interesting that its a new Twitter toy, but I just don't get it. Functional?” programwitch

“Puzzled but Entertained” … “Not really got the slightest idea why this is anything other than an interesting folly.” Tom Coates

“What is the point? Beats me.” Russel Heimlich

“I have to say I was absolutely gobsmacked by how utterly pointless it is.” EirePreneur

(The last one is, short of a personal insult, pretty much the harshest thing anyone has ever said about work I've been involved in.)

To address Tom Coates' point, if you're entertained we've done our job. If you're puzzled then maybe we can help explain things better next time. But I don't mind if some of our work is seen as a folly (“an extravagant, frivolous or fanciful building, designed more for artistic expression than for practicality”). Not everything that everyone does has to be useful or profound. (Nevermind that we've personally found Twitter Blocks a useful way to explore the Twitter network in the last few weeks, with frequent remarks of “I didn't know X was on Twitter”).

Jim Bumgardner (aka KrazyDad, author of O'Reilly's Flickr Hacks) addressed the “so what?” response to frivolous work in a blog post called Utility is Overrated a couple of months back. In the comments there's a comparison with Susan Sontag's Against Interpretation. In it, she states that “interpretation is the revenge of the intellect upon art”. I don't want to point at Twitter Blocks and say “art” (the Motorola sponsorship in particular makes that tricky) but I think that people are thinking too hard about things if they're looking for the “point” of it.

I'm not asking that people stop casting a critical eye over what's presented to them, especially when it's being hyped to death and it's commercially branded. It's fine to ask “what's it for?”, especially of new tools or things that aim to improve the efficiency or effectiveness of this or that. But why not also accept that some things might just be for entertainment and ask “am I having fun” once in a while instead of looking for a problem to be solved or an important statement to be read? Some things just are.

Processing Hacks Fixed

Sadly (due to my neglect as sys-admin) processinghacks.com hasn't been accepting new edits/users since I moved hosts a few months ago. I finally got around to fixing it today, so everything should be working now.

The site hasn't quite reached the ambitious heights I had in mind for it when toxi and I first put the wiki together and drafted the contents. Some of the hack suggestions are out of date now, and many of the existing hacks don't work with the latest release yet. That said, I hope to put some time towards kicking it into shape over the upcoming weeks and I'd be delighted if you'd join me.

I should also mention that there's a good incentive to get your hacks written up, because we're planning to move Processing Hacks to processing.org - you might have seen a note about that in the Learning section. Fame and fortune (or at least some geeky glory) await!

A quick (less certain) note on using Lucene in Processing

In the spirit of continuing our impromptu database week on Processing Blogs (my post, toxi's post, then Florian Jennet updating the sql library), I thought I'd post another quick example using Lucene.

Last week Ryan and I needed a reliable way to search inside a data set we were working with. I had previously tried and failed to write my own useful search routine for the same data, so I wanted to take a look at Lucene instead. (This week, I might have used SQLite, but I hadn't tried it last week!).

Lucene isn't a relational database like MySQL or SQLite, although it has a few similarities with the way most database engines speed up queries using indexes. That's because Lucene is "just" the indexing part. You tell Lucene about your data, one part at a time, and then construct queries to ask it which parts of your data match the query. The key thing is that you keep hold of your data yourself and structure it any way you like, Lucene keeps its own representation of the data for searching. Because of this it can index much more data than you can store in memory.

Anyway, I put up an simple example Lucene applet here that indexes the text of Time Machine by H.G. Wells from Project Gutenberg and lets you type queries against it. The text itself and the Lucene index it builds are both quite small (tens of KB, compressed), but the applet is around 750KB. This is because Lucene's core jar file is about 500KB so it's more suited to standalone projects and applications.

The code is kind of documented, but Lucene was really too much for me to understand properly in just one day. Nevertheless, again, I hope people find it useful!

A Quick Note on using SQLite in Processing

SQLite is a great standalone SQL database engine - not ideal for every situation (particularly large websites), but more than good enough to have already made its way into desktop projects from Apple, and more recently Adobe and Google.

My colleague Mike recently used it as a way to distribute data from a 511.org transit website scraping project he's been doing. I wanted to see if I could download his data (gathered and processed using python) and access it using Processing.

Web searches for SQLite and Java (java wrapper sqlite, etc.) turn up lots of matches, including this promising tutorial from Tim Anderson, but sadly the most prominent matches didn't work for me. It's a real pain to get up and running and installed in a reliable way, mainly due to the need to compile SQLite natively for each platform and then talk to the native code using Java.

Enter this amazing project, a JDBC library for SQLite written in pure java that uses NestedVM to compile the C code for SQLite into something any Java VM can use. It's frightening to think about the amount of misdirection, abstraction and interpretation going on here, but it worked first time and plenty fast enough for my purposes.

Take a look after the jump for the code I ended up with. I hope people find it useful:

(more...)

← Before