Random Etc. Notes to self. Work, play, and the rest.
Visualizing Urban Data: A Journey Through Oakland Crimespotting
Liz Goodman recently invited Mike and I to speak at the UC Berkeley School of Information. We took the opportunity to give a full length talk about a single project, Oakland Crimespotting, which is something of a rarity since we normally try talk about lots of things a little bit, rather than one thing in depth.
Mike started and finished the talk with an in-depth look at the motivations, technical details and social issues surrounding the site, which you can read about on his blog. In the middle I gave a brief overview of related projects and talked about the how the site sits alongside our other mapping work at Stamen. Mike suggested I use a reverse-chronological narrative structure that he liked from a book about Polish history, so I started with the stuff we've finished recently and working back to Stamen's early work with MoveOn and Mappr.
Mike has since reprised the talk for the journalism school, and the whole hour is up on Youtube (part 2 here) if you have time to watch it. Alternatively, you can attempt to simultaneously read the full version of this post and Mike's post together to get a wordier overview of what we talked about.
[At this point Mike has briefly introduced the Oakland Crime site and flash map, and hands over to me for related projects and studio context]
W(e are )here: Mapping The Human Experience
Eric is in Minneapolis at the moment talking about our work at the University of Minnesota. The talk has been in the works for a while but nicely coincides with W(e are )here, and exhibition we're participating in organised by Solutions Twin Cities.
We've prepared a special version of Trulia Hindsight for the show, using the experimental version of Modest Maps I made for Processing in February and animating data for around 1 million homes using OpenGL. We're not ready to distribute the data to a wider audience yet, but here's an example animation from the application:
Trulia Hindsight - Twin Cities Edition from Stamen on Vimeo.
Thanks to Jamie from Trulia for getting us the data we needed to present Trulia Hindsight in this way.
ArtNano (notes on Processing for design elements)
As I mentioned in my previous post, we were working with Geraldine Sarmiento on the ArtNano site, and she came up with the pixellised imagery you see throughout.
Towards the end of the project, I needed a few more of the pixelly images at short notice, to illustrate the essays and about pages on the site. Rather than bother Geraldine, I reached for Processing to see if I could match the look of the homepage imagery that she had created.
I came up with an applet that used a bit of blurring, a bit of distance fall-off, and a bit of perlin noise to create the effect that we were looking for (decorative, obscured, but related to the overall site). Here's an example of the imagery created from a picture of Scott Snibbe's Three Drops that we used on the page for Jennifer Frazier's essay:
You can see the full applet and source code here, and I think it's a good example of how a generative solution to design elements can keep a project flexible right up until launch (and beyond).
ArtNano (notes on Wordpress as a CMS)
I recently finished helping out on the back-end of the NISE Network's ArtNano site for San Francisco's Exploratorium. Working on a 'straight-up' website is a rare departure for us at Stamen, but we're big fans of the Exploratorium and were delighted at the opportunity to work with them.
The ArtNano site features contributions from several artists tasked with exploring the nanoscale. My favourites are Semiconductor's gorgeously glitchy 200 Nanowebbers video, Scott Snibbe's multi-scale Three Drops installation and Santiago Ortiz's Time Spiral. Also featured are artworks by Eric Heller, Victoria Vesna and Stephanie Maxwell.
If you're interested in that sort of thing, read the full version of this post for some notes on using the Wordpress blog platform as a CMS for a small website project like ArtNano.
mySociety Travel Time and House Price Maps
O'Reilly Radar has the scoop on the most recent thing I've finished working on at Stamen. Interactive travel time and house price maps for London. Go play, and read what mySociety have to say, including the ones for BBC TV Centre and the Olympic Stadium site. Then come back and read this full post if you want the background info...
I've had a slow debate running with mySociety's Tom Steinberg since Euro FOO '06 about the best way to present travel time mapping, after we compared notes from my travel time tube map for London and the work Chris Lightfoot did on mapping transport travel times in the UK as a whole.
It seemed like the best way to settle this debate would be for mySociety and Stamen to work together updating their maps and see if we could get the best of both worlds. MySociety had comprehensive travel time data that they, uh, acquired from Transport Direct and Transport for London's journey planners. They also had an ace up their sleeves with the purchase of house price data for London from the UK's land registry. So far the volunteer-led map design had come up with strong proofs of concept, but a consistent set of presentation material was needed to make a compelling argument about the usefulness of these maps in the general case.
Our initial attempts to update the map began with a couple of days of Tom Steinberg and I alternately hunched over Photoshop and poring over the Edward Tufte books (passages recommended by the man himself) and looking for an appropriate colour gradient to represent travel contours (I didn't know about this resource at the time). All the while Francis Irving was back in the UK working on the heavy lifting behind the project, getting the data gathering and overlay rendering up to speed. We tried a few different things, but actually I thought that the palette Tom had worked on with mySociety volunteer Richard Pope was pretty good, and that's the one we used for these static contour maps. (Early mySociety results on the left, updated one in the middle, my final one on the right).
These maps got much less satisfying with the addition of the house price zones, however. That was what Tom had originally asked us to help with; a thorny problem that needed a new approach. After poking around with the various two and three colour overlays with my colleague Mike it became clear that the combinations of masks and outlines were immensely confusing to work with. In addition to this problem, Tom and Francis at mySociety wanted to be flexible about the most appropriate configuration of house prices and travel times to tell a convincing story about the data later. My initial plan had been to set up a workflow in Photoshop to quickly produce the image files they needed. As the complexity of that task became clear, it got less and less attractive.
I had been resisting reaching for a programming solution to the design and cartography problem, but it became clear the manual processing was going to be onerous and difficult to keep consistent. So I asked Francis to provide me with a house price overlay that displayed small price increments using a grey-scale gradient, instead of solid colours between large price bands. Then whilst Tom Steinberg was out at a meeting I cooked up a quick slider experiment in flash to see if the approach had merit, and the difference was astounding. From that moment I was hooked on giving everyone access to what we were playing with, even if it meant working on it over Stamen's christmas break (time/budget constraints had already excluded building a slippy Google-Maps-style map, for example).
Here's a picture of what the house price masks looked like at that point, two colour bands on the left (constrained, harsh), and the new small price increment one (cloudy, beautiful) on the right:
The next task focused on getting a consistent set of base maps for the work, rather than using a hotch-potch of Ordnance Survey maps. The OS maps are world-class for accuracy, of course, but the cartography changes radically at each different scale and the maps aren't designed for on-screen viewing, let alone for data presentation. Naturally (given my history with the project) we turned to OpenStreetMap as a data source and asked Nick Black from ZXV to help produce maps that would be a drop-in replacement for the OS maps mySociety had been using so far.
One thing that Nick's involvement got us was separate layers of data (above), which let us get more fancy with the presentation later on, and keep the labels on top of the data. Of course flexibility giveth, and flexibility taketh away: later on, it took me a while to figure out how to get the text rendering in Flash to match the mapnik style Nick was providing us (more on that later).
So, armed with new map layers, gradient overlays and masks for travel time and house prices, I set about creating a Flash piece that Tom and Francis could configure for themselves (and they did). They're tough cookies to please, of course, and getting something ready that we were all happy with (on volunteer time, across an 8 hour time difference) took patience on everyone's part. Looking at the finished pieces I hope you'll agree that it's worth it, and if you happen to be a transport routing expert sitting on a system that could help produce these maps for everyone, working around the issues Francis identified, then I hope you get in touch!
I have to leave this post here for now, but I hope to go into more detail about the Flash code (which is over here, under a BSD license) in a future post.
Stamen ‘07
Eric just posted a review of what we got up to at Stamen in 2007. My second Thanksgiving at the end of November 2007 marked the end of my first year in San Francisco; 2 months after that it still feels new, fresh and exciting.
The variety of projects and clients at Stamen over the last year or so has been extremely satisfying. I probably won't get a chance to write them up thoroughly for myself, but I'm proud to have had a hand (large or small) in all the projects Eric wrote up (and several more besides!).
Here's to 2008!
Accidental Visual Resonance
In my post about the good people at Yahoo's design research group in September I suggested that some of their visualisations remind me of the movie War Games. I love the movie, but I continue to think that certain kinds of accidental visual resonance should be avoided. The 'incoming' visualisations by the good people at Dopplr have this problem too.
Today, Mike sent me the above image that Gem ffffound showing the devastation caused by the Oakland Hills firestorm in 1991. It's shocking, stunning and scary all at once to see so many homes ablaze like that. Mike pointed out that it looks like some of the work from our Trulia Hindsight project at Stamen.
Thankfully I think Mike was referring to the early prototypes I made in Processing using additive blending and a red-through-blue colour range. I've uploaded a movie of one of these prototypes to Vimeo so you can get an idea of what we're talking about:
San Francisco Property Prices, Animated from Stamen on Vimeo.
The fact that certain parts of the movie looked like San Francisco was burning, or being bombed, was definitely a problem we had to avoid for the final piece. It's something I wouldn't want to be thinking about addicentally if I was trying to find out about real-estate in the area. What we want is to make something that can illustrate the effects of real devastation if we want it to, without emotionally swindling you if you just want to think about urban growth. That's why we knocked out the red and orange hues in the colour range, added a drop shadow and ditched the additive blending. Ultimately, it was more appropriate to show data on the map than in the map.
So, if you want to you can look up some of the areas of Oakland affected by the fires in 1991, such as this example, and spot the clear rebuilding activity in 1992. With luck, the animation will illustrate some of the devastation caused by the fires, without looking like a simulated disaster.
More Musings on Uselessness
Blocks has been out almost a week and the dust is settling a little. Off the cuff remarks are fading out and thoughtful responses are emerging. The second post on Visual Methods, a new blog from MIT Comparitive Media Studies student mike_d, has some good feedback.
I won't address his specific criticisms of Blocks here because agree with a lot of them and we're working on improvements that should go some way to addressing them. However, I do take issue with beginning criticism of the piece using the "standards of canonical information visualization". This relates to the uselessness posts Mike and I made earlier in the week.
Mike_d calls me out on my Techcrunch comment, suggesting that I want both useful and useless at the same time:
No offense to Tom, but it sounds like he's playing both sides! He admits to its potential uselessness but simultaneously suggests that that it is quite functional. Based on Stamen's previous work, I do think they are trying to produce useful information tools and not just pretty designs, but justifications like this seem like an easy way out of more careful consideration of their design. And frankly, the "works for me" defense seems completely antithetical to the principles of information visualization!
I really don't think I'm playing both sides here. Unfortunately, when I respond to people who think Blocks is useless, I'm conceding terms and fighting an uphill battle to make my real point. Perhaps that's a mistake (and yet here I am again).
I think that people who insist on evaluating Blocks as a tool - and conclude that it's useless - are wrong: I've found it useful myself, hence the "worksforme" comment on Techcrunch. But what I really think is that they're looking too hard for the purpose and utility of it before appreciating that it's really a way to look at Twitter from a different angle. I don't think it needs to start from a position of usefulness to be interesting. Furthermore, it's not that it is/isn't useful, it's whether utility is the guiding concern. For us (with Blocks) it wasn't.
(image by Mike, illustrating my words, as a response to Alexandra Deschamps-Sonsino's post)
At Stamen we're knowingly operating on a fine line between aesthetic-driven visualisation (effective information visualisation that is also beautiful) and data-driven aesthetics (beautiful things that also represent data). It's a largely unexplored line (perhaps the area that Mike_d's first post has identified as emerging) that I would compare to the line between designer/coder that I comfortably and steadfastly occupy every day.
Fundamentally though, Blocks is a visualisation of Twitter and its users for Twitter and its users, by Twitter and its users. We have made an attempt to make it understandable for people who don't use the service, but that's a hard task and it's confounded if you're looking for something that's not there.
There aren't really any metrics there, just the things that people said in the order that they said them. Squared.
Criticism for Twitter Blocks
We finally kicked Twitter Blocks out of the door yesterday. It's been in development for about a month, mainly by Ryan Alexander and myself (but all Ryan whilst I was working on Oakland Crimespotting, he's a star). It's the first time we've done 3D things using Flash and it's amazing what's being done with Papervision3D at the moment. On the other hand, having 3D shoe-horned into the 2D Flash engine means the learning curve is a lot steeper than the native-3D Processing/OpenGL worlds Ryan and I are used to.
Whenever Stamen launches a new thing, my immediate reaction is a sigh of relief followed by a slightly obsessive-compulsive trawl of what people are saying about it online. (I use a combination of alerts from Technorati, Google Blog Seach and Bloglines to keep track).
The interesting thing about working with companies like Digg and Twitter is that your work inherits all the criticisms and detractors of those sites as well. Digg's users are clamouring for a picture section, so when we launched Digg Arc many of the responses ignored the piece entirely and chastised Digg for paying attention to visualisation and not to the main site. The same argument is already being used against Twitter Blocks, even though the amount of time Twitter's developers put into it was tiny compared to the amount of time they're putting into stability and new features.
Don't get me wrong: some of the early feedback we're getting is very positive, the team at Twitter have been very receptive and we're proud of our work. This much is good. Some of the feedback we're getting points out that the work isn't immediately understandable (I agree, and maybe we could do some more explaining, but I think we're OK for now).
However, there is also a strong and steady flow of negative comments that I've gathered here so I can think about them all in one place.
“Pretty visualization but I doubt its practicality.” PoppuPot
“Twitter Blocks is the kind of thing that demos well at conferences. Not too useful in real life.” Dave Winer
“exploring myneighbourhood : fun 3D view but so what? not sure i will do that everyday.” jean-michel gobet
“Well its interesting that its a new Twitter toy, but I just don't get it. Functional?” programwitch
“Puzzled but Entertained” … “Not really got the slightest idea why this is anything other than an interesting folly.” Tom Coates
“What is the point? Beats me.” Russel Heimlich
“I have to say I was absolutely gobsmacked by how utterly pointless it is.” EirePreneur
(The last one is, short of a personal insult, pretty much the harshest thing anyone has ever said about work I've been involved in.)
To address Tom Coates' point, if you're entertained we've done our job. If you're puzzled then maybe we can help explain things better next time. But I don't mind if some of our work is seen as a folly (“an extravagant, frivolous or fanciful building, designed more for artistic expression than for practicality”). Not everything that everyone does has to be useful or profound. (Nevermind that we've personally found Twitter Blocks a useful way to explore the Twitter network in the last few weeks, with frequent remarks of “I didn't know X was on Twitter”).
Jim Bumgardner (aka KrazyDad, author of O'Reilly's Flickr Hacks) addressed the “so what?” response to frivolous work in a blog post called Utility is Overrated a couple of months back. In the comments there's a comparison with Susan Sontag's Against Interpretation. In it, she states that “interpretation is the revenge of the intellect upon art”. I don't want to point at Twitter Blocks and say “art” (the Motorola sponsorship in particular makes that tricky) but I think that people are thinking too hard about things if they're looking for the “point” of it.
I'm not asking that people stop casting a critical eye over what's presented to them, especially when it's being hyped to death and it's commercially branded. It's fine to ask “what's it for?”, especially of new tools or things that aim to improve the efficiency or effectiveness of this or that. But why not also accept that some things might just be for entertainment and ask “am I having fun” once in a while instead of looking for a problem to be solved or an important statement to be read? Some things just are.
Oakland Crimespotting
Mike and I have been working on this all week, and it's time to let people see it.
Hopefully the site is self-explanatory, because now I have to work on all the things I wasn't working on so we could get it out there!
← Back to Home
© Random Etc.. Powered by WordPress using the DePo Skinny Theme.