Random Etc. Notes to self. Work, play, and the rest.

mySociety Travel Time and House Price Maps

O'Reilly Radar has the scoop on the most recent thing I've finished working on at Stamen. Interactive travel time and house price maps for London. Go play, and read what mySociety have to say, including the ones for BBC TV Centre and the Olympic Stadium site. Then come back and read this full post if you want the background info...

mySociety travel time map for SW1P 4DR

I've had a slow debate running with mySociety's Tom Steinberg since Euro FOO '06 about the best way to present travel time mapping, after we compared notes from my travel time tube map for London and the work Chris Lightfoot did on mapping transport travel times in the UK as a whole.

It seemed like the best way to settle this debate would be for mySociety and Stamen to work together updating their maps and see if we could get the best of both worlds. MySociety had comprehensive travel time data that they, uh, acquired from Transport Direct and Transport for London's journey planners. They also had an ace up their sleeves with the purchase of house price data for London from the UK's land registry. So far the volunteer-led map design had come up with strong proofs of concept, but a consistent set of presentation material was needed to make a compelling argument about the usefulness of these maps in the general case.

Our initial attempts to update the map began with a couple of days of Tom Steinberg and I alternately hunched over Photoshop and poring over the Edward Tufte books (passages recommended by the man himself) and looking for an appropriate colour gradient to represent travel contours (I didn't know about this resource at the time). All the while Francis Irving was back in the UK working on the heavy lifting behind the project, getting the data gathering and overlay rendering up to speed. We tried a few different things, but actually I thought that the palette Tom had worked on with mySociety volunteer Richard Pope was pretty good, and that's the one we used for these static contour maps. (Early mySociety results on the left, updated one in the middle, my final one on the right).

mySociety map options

These maps got much less satisfying with the addition of the house price zones, however. That was what Tom had originally asked us to help with; a thorny problem that needed a new approach. After poking around with the various two and three colour overlays with my colleague Mike it became clear that the combinations of masks and outlines were immensely confusing to work with. In addition to this problem, Tom and Francis at mySociety wanted to be flexible about the most appropriate configuration of house prices and travel times to tell a convincing story about the data later. My initial plan had been to set up a workflow in Photoshop to quickly produce the image files they needed. As the complexity of that task became clear, it got less and less attractive.

I had been resisting reaching for a programming solution to the design and cartography problem, but it became clear the manual processing was going to be onerous and difficult to keep consistent. So I asked Francis to provide me with a house price overlay that displayed small price increments using a grey-scale gradient, instead of solid colours between large price bands. Then whilst Tom Steinberg was out at a meeting I cooked up a quick slider experiment in flash to see if the approach had merit, and the difference was astounding. From that moment I was hooked on giving everyone access to what we were playing with, even if it meant working on it over Stamen's christmas break (time/budget constraints had already excluded building a slippy Google-Maps-style map, for example).

Here's a picture of what the house price masks looked like at that point, two colour bands on the left (constrained, harsh), and the new small price increment one (cloudy, beautiful) on the right:

mySociety house price mask comparison

The next task focused on getting a consistent set of base maps for the work, rather than using a hotch-potch of Ordnance Survey maps. The OS maps are world-class for accuracy, of course, but the cartography changes radically at each different scale and the maps aren't designed for on-screen viewing, let alone for data presentation. Naturally (given my history with the project) we turned to OpenStreetMap as a data source and asked Nick Black from ZXV to help produce maps that would be a drop-in replacement for the OS maps mySociety had been using so far.

OSM layers for mySociety maps

One thing that Nick's involvement got us was separate layers of data (above), which let us get more fancy with the presentation later on, and keep the labels on top of the data. Of course flexibility giveth, and flexibility taketh away: later on, it took me a while to figure out how to get the text rendering in Flash to match the mapnik style Nick was providing us (more on that later).

So, armed with new map layers, gradient overlays and masks for travel time and house prices, I set about creating a Flash piece that Tom and Francis could configure for themselves (and they did). They're tough cookies to please, of course, and getting something ready that we were all happy with (on volunteer time, across an 8 hour time difference) took patience on everyone's part. Looking at the finished pieces I hope you'll agree that it's worth it, and if you happen to be a transport routing expert sitting on a system that could help produce these maps for everyone, working around the issues Francis identified, then I hope you get in touch!

I have to leave this post here for now, but I hope to go into more detail about the Flash code (which is over here, under a BSD license) in a future post.


19 Comments

Wow, setting the interval to 5 or 10 minutes and then translating the slider creates an effect not unlike watching palettes cycle on the old MS-DOS fractal creation programs I used to obsess over…

Posted by Darius K. on 24 January 2008 @ 10pm

Indeed - palette cycling would probably have been a faster way to do this ;)

Posted by TomC on 25 January 2008 @ 12am

i’m working on something similar at the moment (based on the wonderful modestmaps). one of the problems i encountered is the poor performance of rendering the map overlay.
as of now i’m using a modified version from seld.be: http://seld.be/code/heatmap
i tried a similar approach to the one you used (as far as i can tell, just skimmed parts of the code): drawing on a bitmap in grayscale gradients (draw on sprite using drawCircle, fill bitmap with sprite), filter for low, mid and high values of gray (ColorTransform) and then tinting the image. i got stuck at that point and reverted to the seld.be code.
now it takes a whooping 1-4s to draw something like 80,000 datapoints over the map. i’m thinking about hooking the drawing up to one of the tile request functions in modestmaps to render only small amounts of data per tile and then let modestmaps figure out where to place them.

i’ll try to work out how you did it and incorporate your code to see how it performs for me.

Posted by adrian on 25 January 2008 @ 8am

erm. s/ColorTransform/ColorTransform
http://maso.lin.googlepages.com/ColorMatrixTool.html provided a nice interface to figure out the correct values (ColorMatrix is a black box to me ;-) )

Posted by adrian on 25 January 2008 @ 8am

Great to see the maps in the wild. Just a quick mention of the help that we had at ZXV from Artem Pavlenko, whose Mapnik rendering software produced the underlying maps.

Posted by Nick on 25 January 2008 @ 10am

Adrian - 80,000 data points is a lot of stuff to draw in Flash at one time… interactively anyway. For instance, even in AS3 we limit the data to 5000 points in Trulia Hindsight.

Rather than drawing sprites onto the bitmap one at a time, you might try copying bitmaps onto the bitmap, that could be faster?

But yes, ultimately you’ll probably end up pre-rendering bitmaps, as Francis at mySociety did for this project. Hooking those up to a tile server would be a nice next step.

Posted by TomC on 25 January 2008 @ 9pm

i was stumped by bitmapData.threshold - somehow i never realized what it was good for. the plan - atm - is to hook a overlay function into the tile request process of modestmaps (yay! 1.0 release.) and get the data from a local server. to reduce rendering time i thought of clustering the data points depending on the zoom level. all this should give me nice, small sets of data to handle.

pre-rendering the tiles and provide them as bitmaps won’t work - the data is too “dynamic”. we’ve got time, location, user & rating. perhaps hooking a bit of predective logic into the app and use some idle time of the client might work.

Posted by adrian on 26 January 2008 @ 11am

[...] Interactive Travel Time and House Price Maps - Tom from Stamen, recently announced some really slick mapping. They’re very attractive and very responsive. Sidenote: Look forward to a guest post from Tom in the near future. [...]

Posted by Weekend Minis - Maps, Motion & Resources on 26 January 2008 @ 11pm

If you want to do some pseudo palette mapping with some colors try BitmapData.paletteMap instead of BitmapData.threshold. Using it wisely you can convert a greyscale image (like the one you have) into a discrete-colored image. It worked great for me.

Posted by marco q on 28 January 2008 @ 3pm

Just an off-the-cuff thought for the travel-time versus home price map. You could tint areas yellow that had travel time under the criterion, and tint areas blue that had home prices under the criterion, and if you made the two colour overlays transparent, the areas meeting both criteria would be green. Green is pretty universally ‘good’ so the result would be unambiguous and useful. But would the use of transparency ruin performance? No idea.

Posted by Dominic Brown on 28 January 2008 @ 11pm

[...] Travel Time Maps and other projects. Comments on travel time maps from co-creator Tom [...]

Posted by Meryl.net » 175+ Data and Information Visualization Examples and Resources on 28 January 2008 @ 11pm

Dominic - it wasn’t really about the performance, the final Flash piece is pretty close to something we were happy with in Photoshop without reference to whether it was even possible in Flash.

Tinting the map is something that was considered, but ultimately I wanted to make the thing be a map of the places that you can live. Given that, it seems to me that for the ‘yes+yes’ places, the original cartography should be preserved. The final piece does tint the ‘no’ places to varying degrees though, and that tinting is multiplicative like you suggest.

Posted by TomC on 29 January 2008 @ 12am

[...] the mySociety travel time maps I wanted to have user-defined labels that looked pretty much like the map styles in the [...]

Posted by Random Etc. » Blog Archive » Simple Text Outlines in Flash on 31 January 2008 @ 6am

[...] mySociety Travel Time and House Price Maps [...]

Posted by Workshop session on geodata visualization at Alper.nl on 14 June 2008 @ 2pm

[...] Random Etc. - mySociety Travel Time and House Price Maps Tom Carden talks about his work on the MySociety travel-time maps, which I really like. (tags: maps tomcarden stamen mysociety traveltimes) [...]

Posted by Leapfroglog - links for 2008-07-31 on 30 July 2008 @ 10pm

[...] Travel Time Maps and other projects. Comments on travel time maps from co-creator Tom [...]

Posted by Inforgraphics Examples and Resources | LLYMOTION DESIGN NOTES on 16 September 2010 @ 6am

[...] prints and embeds of the travel time isochrone maps. It’s been a long time coming, and working with Tom Carden on the ways in which the data and display layers fit together was a really foundational time for [...]

Posted by HQ2 Week 96 | Stamen Design on 27 July 2012 @ 3pm

[...] maps I talked about in my talk today are online here, and Tom has done a very nice writeup on his blog. I haven’t had time to officially put the project on Stamen yet, and a few people have asked [...]

Posted by Two Talks | Stamen Design on 27 July 2012 @ 5pm

[...] Our newly released travel time maps are currently shooting round the internet. It was great fun making them, and you might like to have a go too – there are plenty of public datasets you could overlay on the same base maps, using the same flash app (source code). There are a few notes about how we made them on the page itself, and the associated real time page. For a far more interesting view of the development process, read Tom Carden from Stamen’s account. [...]

Posted by Here’s to a shorter commute / mySociety on 3 April 2014 @ 4am