Eric is in Minneapolis at the moment talking about our work at the University of Minnesota. The talk has been in the works for a while but nicely coincides with W(e are )here, and exhibition we're participating in organised by Solutions Twin Cities.

We've prepared a special version of Trulia Hindsight for the show, using the experimental version of Modest Maps I made for Processing in February and animating data for around 1 million homes using OpenGL. We're not ready to distribute the data to a wider audience yet, but here's an example animation from the application:

Trulia Hindsight - Twin Cities Edition from Stamen on Vimeo.

Thanks to Jamie from Trulia for getting us the data we needed to present Trulia Hindsight in this way.

So:

Modest Maps is a BSD-licensed display and interaction library for tile-based maps in Flash (ActionScript 2.0 and ActionScript 3.0) and Python...

...And Processing

The greatest thing about Google Maps, and the follow-up hit album API, is that it made a big song and dance about the concept of continuously scrolling maps. So now everyone has to have one. Google, Yahoo, Microsoft and many other big companies have server farms full of tiled maps of varying zoom levels, and free-to-use javascript APIs to let you put your own data on top of them. Sadly, they don't all offer a way to use those same images in Flash applications, which is what prompted my colleague Mike and friends to write the original Actionscript 2 version of Modest Maps (and our subsequent port to AS3 and Flex) nor is it possible to use the map images server-side, which prompted the port to Python. Python is also Mike's thinking language of choice. Mine is Processing, so improving Modest Maps has always been something of an uphill struggle for me. (I've also been in this territory once before, with Mapstraction, so I was reluctant to dive in again with the tool-making).

Because of restrictions on how Java applets load images from third-party domains, a Processing applet won't be able to access anyone else's image servers directly. I've resisted porting/implementing Modest Maps in Processing until now for that reason. However, thanks partly to my renewed interest in Processing and OpenGL, and thanks partly to a desire to re-implement the Actionscript 3 version of Modest Maps to take better advantage of matrix transforms, I've finally caved in and worked on it.

To the matrix transformation end, I've knocked together a little sketch that shows how to scroll around a tiled world that uses the same scheme as Google Maps: numbered zoom levels, rows and columns. Use the mouse to drag and the +/- keys to zoom in and out. Rows and columns start at zero in the top left, zoom level 0 has 1 row and column, zoom level 1 has 2, zoom level 2 has 4, zoom level 3 has 8, and so on in powers of 2 such that zoom level 17 has 131,072 rows and columns. At zoom level 17 that means there are (131,072²) or 17,179,869,184 tiles in the whole world, if you render them all (and you probably wouldn't)!

Once that bit is figured out, it's a case of getting hold of an image for each tile, and the logic for this is most succinctly expressed in the Modest Maps Python source code. I've ported that over to Processing with as few changes as possible, and a bit of testing and bit more thinking later I'm ready to offer up a test application (mac, windows, linux) to show what it's like so far:

There are some things that this version does, that the Flash one doesn't:

• it has reliable continuous scaling between zoom levels (use the mouse-wheel or press +/-, press 'z' to snap to the nearest pixel-perfect level)
• it tries to show the best image that it has for each zoom level (going up to one level in or as many levels out as it takes)
• it tries to load tiles from the centre of the map out to the edges

Sadly it also currently redraws everything on every frame (for 100% CPU Usage™, ask for it by name) – that will have to be fixed if we port this code back to Flash.

The Code

I'm not quite ready to distribute and support a library, but if you're not comfortable with grabbing the code from subversion and compiling for yourself, here's a zip file with a .jar file for your code folder and an example .pde file to get you started. I've thrown in some buttons too, no extra charge! The API is largely the same as the Actionscript 3 version: panning and zooming are solid, and it's easy to get the lat/lon location of points on the screen, and the point on the screen for lat/lon locations.

Future

Better documentation, animation (perhaps with Lee Byron's ShapeTween), event handling and such will have to wait for a future version: contributions and bug-fixes are welcome, feel free to leave a comment here if you'd like to get in touch.

The Final Word

Nobody from Google, Microsoft or Yahoo has given me, or you, permission to use their images within Processing. Apart from the fact that it won't work as an applet anyway, I should state clearly that I'm really only offering this library for personal use, and that it's covered by the same BSD license as the rest of Modest Maps, which comes with absolutely no warranties or guarantees. Caveat Coder!

I've put some code about line caps and joins in Processing & OpenGL up on Processing Hacks in case it's something that bothers you too. I've not tested it extensively so I'd welcome bug fixes and suggestions there, or in the comments here. One thing I'm interested in doing next is extending (or reimplementing) BasicStroke to generate shapes for thick polylines with varying line thickness. If that's something you've done before, please let me know.

I'm using Java's BasicStroke to generate the path outlines, and GLU's tesselators to generate a mesh that will fill correctly in OpenGL. I "borrowed" the code for the latter from Ben Fry's PGraphicsOpenGL font rendering – thanks Ben!

Towards the end of the project, I needed a few more of the pixelly images at short notice, to illustrate the essays and about pages on the site. Rather than bother Geraldine, I reached for Processing to see if I could match the look of the homepage imagery that she had created.

I came up with an applet that used a bit of blurring, a bit of distance fall-off, and a bit of perlin noise to create the effect that we were looking for (decorative, obscured, but related to the overall site). Here's an example of the imagery created from a picture of Scott Snibbe's Three Drops that we used on the page for Jennifer Frazier's essay:

You can see the full applet and source code here, and I think it's a good example of how a generative solution to design elements can keep a project flexible right up until launch (and beyond).

That was the rationale behind our Trulia Hindsight movies (show something about everything) and our charts of a day of activity on Digg. It's definitely an approach I've found easiest in Processing, although using it recently I've missed the instant mouse-driven interactivity of Flash or HTML.

My colleague Shawn just posted a visualisation he made to help debug a visualisation he's working on at the moment:

Shawn's chart is basically a simple scatter plot of cabspotting data points (by cab ID and time), except that he's also overlaid some of the connections between the data to show how far back and forward he has to look to accurately predict a cab's location. And the whole thing moves beautifully, showing up bad data and highlighting good data as it goes. Hopefully we'll get a video up soon.

In the meantime, be sure to read Shawn's description and keep an eye out for the final debugged visualization in the MOMA soon!

Today, Mike sent me the above image that Gem ffffound showing the devastation caused by the Oakland Hills firestorm in 1991. It's shocking, stunning and scary all at once to see so many homes ablaze like that. Mike pointed out that it looks like some of the work from our Trulia Hindsight project at Stamen.

Thankfully I think Mike was referring to the early prototypes I made in Processing using additive blending and a red-through-blue colour range. I've uploaded a movie of one of these prototypes to Vimeo so you can get an idea of what we're talking about:

San Francisco Property Prices, Animated from Stamen on Vimeo.

The fact that certain parts of the movie looked like San Francisco was burning, or being bombed, was definitely a problem we had to avoid for the final piece. It's something I wouldn't want to be thinking about addicentally if I was trying to find out about real-estate in the area. What we want is to make something that can illustrate the effects of real devastation if we want it to, without emotionally swindling you if you just want to think about urban growth. That's why we knocked out the red and orange hues in the colour range, added a drop shadow and ditched the additive blending. Ultimately, it was more appropriate to show data on the map than in the map.

So, if you want to you can look up some of the areas of Oakland affected by the fires in 1991, such as this example, and spot the clear rebuilding activity in 1992. With luck, the animation will illustrate some of the devastation caused by the fires, without looking like a simulated disaster.

Instead of Feedwordpress and Planet Planet put together with sticky tape and glue, I'm using the better documented Planet Venus and WP-Venus so our archives are maintained.

If you ever need to run a site similar to Processing Blogs and your web host can run python then I definitely recommend Planet or Planet Venus as the solution. WP-Venus complained a little bit when I converted from Feedwordpress, but it looks good so far: hopefully the archives will be worth the effort.

Feeds are now being grabbed robustly by Planet and merely massaged by Feedwordpress. Here's the current list:

• Grow Up!
• abstractmachine
• all manner of distractions
• blog.blprnt.com - Processing
• Code & Form
• Create Digital Motion
• daniel shiffman
• Digital Tools
• eskimoblood
• fladdict.net
• i.document
• Ivan Safrin's Code Sketchbook
• Lightcycle
• m a n i a x | m e m o r i
• marumushi.com works
• Metaphorical.net - powered by FeedBurner
• nuthinking
• Pixelsumo » Processing
• Random Etc.
• RobotAcid
• Scrapbook
• Setpixel
• steve cooley presents » processing.org
• Software over the rainbow
• The Sound of Crickets Chirping
• thinking
• toxi.in.process
• v3ga
• YouTube :: Tag // processing.org
• Processing.org Updates
• Processing.org Happenings
• generative.grfx / Generative Graphics

Please do let me know if I'm syndicating too much or too little, and especially if I'm missing people or if I'm syndicating your blog and you don't want me to. Otherwise, the site should run itself for a while.