Random Etc. Notes to self. Work, play, and the rest.
Visualizing Urban Data: A Journey Through Oakland Crimespotting
Liz Goodman recently invited Mike and I to speak at the UC Berkeley School of Information. We took the opportunity to give a full length talk about a single project, Oakland Crimespotting, which is something of a rarity since we normally try talk about lots of things a little bit, rather than one thing in depth.
Mike started and finished the talk with an in-depth look at the motivations, technical details and social issues surrounding the site, which you can read about on his blog. In the middle I gave a brief overview of related projects and talked about the how the site sits alongside our other mapping work at Stamen. Mike suggested I use a reverse-chronological narrative structure that he liked from a book about Polish history, so I started with the stuff we've finished recently and working back to Stamen's early work with MoveOn and Mappr.
Mike has since reprised the talk for the journalism school, and the whole hour is up on Youtube (part 2 here) if you have time to watch it. Alternatively, you can attempt to simultaneously read the full version of this post and Mike's post together to get a wordier overview of what we talked about.
[At this point Mike has briefly introduced the Oakland Crime site and flash map, and hands over to me for related projects and studio context]
W(e are )here: Mapping The Human Experience
Eric is in Minneapolis at the moment talking about our work at the University of Minnesota. The talk has been in the works for a while but nicely coincides with W(e are )here, and exhibition we're participating in organised by Solutions Twin Cities.
We've prepared a special version of Trulia Hindsight for the show, using the experimental version of Modest Maps I made for Processing in February and animating data for around 1 million homes using OpenGL. We're not ready to distribute the data to a wider audience yet, but here's an example animation from the application:
Trulia Hindsight - Twin Cities Edition from Stamen on Vimeo.
Thanks to Jamie from Trulia for getting us the data we needed to present Trulia Hindsight in this way.
ArtNano (notes on Processing for design elements)
As I mentioned in my previous post, we were working with Geraldine Sarmiento on the ArtNano site, and she came up with the pixellised imagery you see throughout.
Towards the end of the project, I needed a few more of the pixelly images at short notice, to illustrate the essays and about pages on the site. Rather than bother Geraldine, I reached for Processing to see if I could match the look of the homepage imagery that she had created.
I came up with an applet that used a bit of blurring, a bit of distance fall-off, and a bit of perlin noise to create the effect that we were looking for (decorative, obscured, but related to the overall site). Here's an example of the imagery created from a picture of Scott Snibbe's Three Drops that we used on the page for Jennifer Frazier's essay:
You can see the full applet and source code here, and I think it's a good example of how a generative solution to design elements can keep a project flexible right up until launch (and beyond).
ArtNano (notes on Wordpress as a CMS)
I recently finished helping out on the back-end of the NISE Network's ArtNano site for San Francisco's Exploratorium. Working on a 'straight-up' website is a rare departure for us at Stamen, but we're big fans of the Exploratorium and were delighted at the opportunity to work with them.
The ArtNano site features contributions from several artists tasked with exploring the nanoscale. My favourites are Semiconductor's gorgeously glitchy 200 Nanowebbers video, Scott Snibbe's multi-scale Three Drops installation and Santiago Ortiz's Time Spiral. Also featured are artworks by Eric Heller, Victoria Vesna and Stephanie Maxwell.
If you're interested in that sort of thing, read the full version of this post for some notes on using the Wordpress blog platform as a CMS for a small website project like ArtNano.
Step 1: Can We Show It All?
I've been thinking recently about data visualisation approaches. One that I'm very fond of in the early stage of a project is to figure out a way to arrange the whole thing on screen – or a representative sample of it – and figure out what meaningful segments you can mark on top of it.
That was the rationale behind our Trulia Hindsight movies (show something about everything) and our charts of a day of activity on Digg. It's definitely an approach I've found easiest in Processing, although using it recently I've missed the instant mouse-driven interactivity of Flash or HTML.
My colleague Shawn just posted a visualisation he made to help debug a visualisation he's working on at the moment:
Shawn's chart is basically a simple scatter plot of cabspotting data points (by cab ID and time), except that he's also overlaid some of the connections between the data to show how far back and forward he has to look to accurately predict a cab's location. And the whole thing moves beautifully, showing up bad data and highlighting good data as it goes. Hopefully we'll get a video up soon.
In the meantime, be sure to read Shawn's description and keep an eye out for the final debugged visualization in the MOMA soon!
mySociety Travel Time and House Price Maps
O'Reilly Radar has the scoop on the most recent thing I've finished working on at Stamen. Interactive travel time and house price maps for London. Go play, and read what mySociety have to say, including the ones for BBC TV Centre and the Olympic Stadium site. Then come back and read this full post if you want the background info...
I've had a slow debate running with mySociety's Tom Steinberg since Euro FOO '06 about the best way to present travel time mapping, after we compared notes from my travel time tube map for London and the work Chris Lightfoot did on mapping transport travel times in the UK as a whole.
It seemed like the best way to settle this debate would be for mySociety and Stamen to work together updating their maps and see if we could get the best of both worlds. MySociety had comprehensive travel time data that they, uh, acquired from Transport Direct and Transport for London's journey planners. They also had an ace up their sleeves with the purchase of house price data for London from the UK's land registry. So far the volunteer-led map design had come up with strong proofs of concept, but a consistent set of presentation material was needed to make a compelling argument about the usefulness of these maps in the general case.
Our initial attempts to update the map began with a couple of days of Tom Steinberg and I alternately hunched over Photoshop and poring over the Edward Tufte books (passages recommended by the man himself) and looking for an appropriate colour gradient to represent travel contours (I didn't know about this resource at the time). All the while Francis Irving was back in the UK working on the heavy lifting behind the project, getting the data gathering and overlay rendering up to speed. We tried a few different things, but actually I thought that the palette Tom had worked on with mySociety volunteer Richard Pope was pretty good, and that's the one we used for these static contour maps. (Early mySociety results on the left, updated one in the middle, my final one on the right).
These maps got much less satisfying with the addition of the house price zones, however. That was what Tom had originally asked us to help with; a thorny problem that needed a new approach. After poking around with the various two and three colour overlays with my colleague Mike it became clear that the combinations of masks and outlines were immensely confusing to work with. In addition to this problem, Tom and Francis at mySociety wanted to be flexible about the most appropriate configuration of house prices and travel times to tell a convincing story about the data later. My initial plan had been to set up a workflow in Photoshop to quickly produce the image files they needed. As the complexity of that task became clear, it got less and less attractive.
I had been resisting reaching for a programming solution to the design and cartography problem, but it became clear the manual processing was going to be onerous and difficult to keep consistent. So I asked Francis to provide me with a house price overlay that displayed small price increments using a grey-scale gradient, instead of solid colours between large price bands. Then whilst Tom Steinberg was out at a meeting I cooked up a quick slider experiment in flash to see if the approach had merit, and the difference was astounding. From that moment I was hooked on giving everyone access to what we were playing with, even if it meant working on it over Stamen's christmas break (time/budget constraints had already excluded building a slippy Google-Maps-style map, for example).
Here's a picture of what the house price masks looked like at that point, two colour bands on the left (constrained, harsh), and the new small price increment one (cloudy, beautiful) on the right:
The next task focused on getting a consistent set of base maps for the work, rather than using a hotch-potch of Ordnance Survey maps. The OS maps are world-class for accuracy, of course, but the cartography changes radically at each different scale and the maps aren't designed for on-screen viewing, let alone for data presentation. Naturally (given my history with the project) we turned to OpenStreetMap as a data source and asked Nick Black from ZXV to help produce maps that would be a drop-in replacement for the OS maps mySociety had been using so far.
One thing that Nick's involvement got us was separate layers of data (above), which let us get more fancy with the presentation later on, and keep the labels on top of the data. Of course flexibility giveth, and flexibility taketh away: later on, it took me a while to figure out how to get the text rendering in Flash to match the mapnik style Nick was providing us (more on that later).
So, armed with new map layers, gradient overlays and masks for travel time and house prices, I set about creating a Flash piece that Tom and Francis could configure for themselves (and they did). They're tough cookies to please, of course, and getting something ready that we were all happy with (on volunteer time, across an 8 hour time difference) took patience on everyone's part. Looking at the finished pieces I hope you'll agree that it's worth it, and if you happen to be a transport routing expert sitting on a system that could help produce these maps for everyone, working around the issues Francis identified, then I hope you get in touch!
I have to leave this post here for now, but I hope to go into more detail about the Flash code (which is over here, under a BSD license) in a future post.
Stamen ‘07
Eric just posted a review of what we got up to at Stamen in 2007. My second Thanksgiving at the end of November 2007 marked the end of my first year in San Francisco; 2 months after that it still feels new, fresh and exciting.
The variety of projects and clients at Stamen over the last year or so has been extremely satisfying. I probably won't get a chance to write them up thoroughly for myself, but I'm proud to have had a hand (large or small) in all the projects Eric wrote up (and several more besides!).
Here's to 2008!
Accidental Visual Resonance
In my post about the good people at Yahoo's design research group in September I suggested that some of their visualisations remind me of the movie War Games. I love the movie, but I continue to think that certain kinds of accidental visual resonance should be avoided. The 'incoming' visualisations by the good people at Dopplr have this problem too.
Today, Mike sent me the above image that Gem ffffound showing the devastation caused by the Oakland Hills firestorm in 1991. It's shocking, stunning and scary all at once to see so many homes ablaze like that. Mike pointed out that it looks like some of the work from our Trulia Hindsight project at Stamen.
Thankfully I think Mike was referring to the early prototypes I made in Processing using additive blending and a red-through-blue colour range. I've uploaded a movie of one of these prototypes to Vimeo so you can get an idea of what we're talking about:
San Francisco Property Prices, Animated from Stamen on Vimeo.
The fact that certain parts of the movie looked like San Francisco was burning, or being bombed, was definitely a problem we had to avoid for the final piece. It's something I wouldn't want to be thinking about addicentally if I was trying to find out about real-estate in the area. What we want is to make something that can illustrate the effects of real devastation if we want it to, without emotionally swindling you if you just want to think about urban growth. That's why we knocked out the red and orange hues in the colour range, added a drop shadow and ditched the additive blending. Ultimately, it was more appropriate to show data on the map than in the map.
So, if you want to you can look up some of the areas of Oakland affected by the fires in 1991, such as this example, and spot the clear rebuilding activity in 1992. With luck, the animation will illustrate some of the devastation caused by the fires, without looking like a simulated disaster.
More Musings on Uselessness
Blocks has been out almost a week and the dust is settling a little. Off the cuff remarks are fading out and thoughtful responses are emerging. The second post on Visual Methods, a new blog from MIT Comparitive Media Studies student mike_d, has some good feedback.
I won't address his specific criticisms of Blocks here because agree with a lot of them and we're working on improvements that should go some way to addressing them. However, I do take issue with beginning criticism of the piece using the "standards of canonical information visualization". This relates to the uselessness posts Mike and I made earlier in the week.
Mike_d calls me out on my Techcrunch comment, suggesting that I want both useful and useless at the same time:
No offense to Tom, but it sounds like he's playing both sides! He admits to its potential uselessness but simultaneously suggests that that it is quite functional. Based on Stamen's previous work, I do think they are trying to produce useful information tools and not just pretty designs, but justifications like this seem like an easy way out of more careful consideration of their design. And frankly, the "works for me" defense seems completely antithetical to the principles of information visualization!
I really don't think I'm playing both sides here. Unfortunately, when I respond to people who think Blocks is useless, I'm conceding terms and fighting an uphill battle to make my real point. Perhaps that's a mistake (and yet here I am again).
I think that people who insist on evaluating Blocks as a tool - and conclude that it's useless - are wrong: I've found it useful myself, hence the "worksforme" comment on Techcrunch. But what I really think is that they're looking too hard for the purpose and utility of it before appreciating that it's really a way to look at Twitter from a different angle. I don't think it needs to start from a position of usefulness to be interesting. Furthermore, it's not that it is/isn't useful, it's whether utility is the guiding concern. For us (with Blocks) it wasn't.
(image by Mike, illustrating my words, as a response to Alexandra Deschamps-Sonsino's post)
At Stamen we're knowingly operating on a fine line between aesthetic-driven visualisation (effective information visualisation that is also beautiful) and data-driven aesthetics (beautiful things that also represent data). It's a largely unexplored line (perhaps the area that Mike_d's first post has identified as emerging) that I would compare to the line between designer/coder that I comfortably and steadfastly occupy every day.
Fundamentally though, Blocks is a visualisation of Twitter and its users for Twitter and its users, by Twitter and its users. We have made an attempt to make it understandable for people who don't use the service, but that's a hard task and it's confounded if you're looking for something that's not there.
There aren't really any metrics there, just the things that people said in the order that they said them. Squared.
Design.Yahoo: Web 2.0 meets War Games?
Whilst I wrestle with my reaction to the reception of Twitter Blocks, it's interesting to look at what other people in information visualisation are working on.
Yahoo's new design research outfit, apparently also known as yhaus, have just put up a site outlining their work so far. The first thing I noticed is that they've snagged an amazing subdomain: design.yahoo.com. The second thing I noticed is that compared to other teams I'm aware of in the field (including Stamen) they have a pretty good gender balance (a thorny issue but I'm noting it anyway). The third thing I'll note is the guest appearance of non-Yahoo work in the portraits of the team: I see Torrent Raiders, Fidg't... what else?
Sadly, all but one of the demos there so far is a big Quicktime movie. I know that with millions of users Yahoo has to be a stickler for browser support and compatibility, but I hope they get a chance to take this work live on the web as well as demo it in movie form. There's some solid realtime Flash and Processing work hiding in there, and people (OK, I) want to see it in its interactive entirety.
There's clearly some healthy collaboration and influence going on there (much as in our work, e.g. Ben Fry's zipdecode looms large over the interactive version of our Trulia search animation). Yahoo's Aaron Koblin is best known for his Flight Patterns piece, and this visualisation by Michael Chang of Yahoo trip planner data is very similar:
Likewise, Aaron's work on traffic patterns bears a close resemblance to Flight Patterns:
I don't want to pick on them too much, because it's really beautiful work I admire a great deal (and it might seem like sour grapes), but both the pieces I've highlighted do suffer from something we've tried to avoid at Stamen: animated information graphics on top of black backgrounds and vector maps can easily look like screenshots from a modern-day War Games:
I'm glad other people as well as us are experimenting in public, and I'm glad sites like Infosthetics and Visual Complexity are cataloguing our efforts. We need our own visual language around this kind of visualisation that doesn't resonate with the imagery of war.
Apple's iphone has made a strong impression with slick transitions in its interface design, but the maps application still borrows from Google's pseudo-shadows and static pins. The playful interfaces of the Nintendo's Wii games certainly offer a different path, but the rest of the games (and movie) industry's cinematic-realism aesthetic exerts a strong influence over our generation of designers and it isn't going to meet these goals any time soon.
It's a fairly regular topic of conversation at Stamen: how can you make a visualisation of e.g. 911 calls actually look like emergencies, and not birthday parties or toilet flushes, without freaking people out and without making it mundane? Is it possible to use great circles to connect air travel destinations without it looking like missiles? Can you animate growing and shrinking red and yellow circles on an aerial map without it looking like Gulf War I weapons company propaganda?
It's a bloggy weekend here at Random Etc, if you're reading along I'm definitely interested to hear your thoughts.
← Back to Home
© Random Etc.. Powered by WordPress using the DePo Skinny Theme.