A few people have asked if I'm still using Processing now that I've joined Stamen (best known for their Flash work). Whilst it's true that I've been quiet on the Processing front, hard at work learning Actionscript 3 for Flash 9 to enable us to deliver Trulia Hindsight, much of that piece was informed by early sketches I wrote in Processing. (The graphs we made of a day of diggs were also made with Processing.)
Here are four movies we made from Trulia's data to get across the ideas we wanted to develop into Hindsight. It's a lucky thing that Flash 9 can shift many more points around the screen than Flash 8, otherwise we'd have been stuck. That said, it's still way behind Processing with OpenGL for this kind of visualisation, so choose your tools wisely when building a proof-of-concept in a different language to the one your project will be delivered in!
In the first few weeks of working with Trulia, we did some initial work exploring non-geographic views of their data such as tree maps and node graphs and so on. In the end though, the most compelling thing we came up with was to explore the different dimensions of the Trulia database in the form of animated maps.
Here are two movies, the first is San Francisco and the second is San Jose, showing the properties animated along an intuitive axis: the year they were built.
And here are two more movies, also made with Processing, that show the properties that were sold in the last 10 years (under $2m), this time animated by sale price:
The animations by sale price aren't available in Trulia Hindsight (yet) but we hope to work more on these less-intuitive dimensions for animation in the future.
No doubt whichever direction we go in next I'll still reach for Processing to try things out. It's the tool I find it easiest to think in, and although Flex Builder (based on Eclipse) is a great IDE, I still find myself wanting to bend Actionscript to be more like Processing when it comes to prototyping my ideas - it seems I'm not the only one!
If you're "someone who is a designer first and foremost, a coder a distant second, and who's interested in where these areas interleave" then please read the post on our site and consider getting in touch. I can't recommend us highly enough.
Twitter is a website that asks only one thing, "what are you doing?" and aggregates your responses intermingled with the responses of your friends over the last 24 hours. If you let it (I don't) it will SMS you every time your friends update, or if you prefer (I do) it will send you an instant message instead. It will also let you update by web, IM or SMS. It's certainly an easy way to SMS a group of people and only pay for one message, but the IM and web integration mean it's more than just group SMS.
So it's not IM, SMS or the web, but it talks to all three. I like it. I want to hate it. I suppose I cheat, because I don't let it SMS me very often. And maybe because most of my contacts are a continent away, so I only get a few messages a day (they're all asleep). But there it is: I'm not stressed out by it, I'm still Getting Things Done (though that system's not for me, yet). Continuous Partial Attention be damned.
Of course, it's fully Web 2.0 buzz-word compliant, so it has an API that you can use to get data in and out. Not a super-useful one for visualisation, but useful enough to get started. Knocking some ideas back and forth at Stamen with Eric yesterday I decided it was worth trying his idea of plotting twitter activity on a circle. I started with a circle representing the previous 24 hours, rather than a 12 hour clock face, for several reasons:
That's it really.
Given 24 hours of statuses I assigned each user a colour and plotted the status at an angle corresponding to how much of the day had elapsed. I joined each message to the previous message from that person, if there was one. Here is how my first pass turned out:
And here's another variation, still with a colour per person but ditching the arcs and instead using concentric rings for status messages. There are small dots again mapped to time of day. Moire be damned.
The top of the circle is midnight (PST), the bottom is noon. The data was sampled at about 4pm. I'm not sure where this circular/spiral visualisation is going, but if I revisit it I will probably unroll them into a rectangle in the hope that there is space to draw and read the messages. After all, the messages are what it's all about.
These were built with Processing, using the now-built-in XML support and the gorgeous PDF library. I haven't posted the applet because it doesn't work online with the Twitter API, sorry.
A new version of Processing is available now.
If you haven't looked at Processing for a while, it's definitely worth picking up again. It's maturing very nicely - this week I've been using it at work, and the ability to create a single app with full-screen OpenGL graphics, PDF and movie output (thanks to Dan Shiffman's movie maker library) is absolutely great.
The beginning of the acadmic year must be upon us, because Ben Fry and Casey Reas just released the latest round of bug-fixes and updates to the Processing libraries and development environment. The changes are meticulously documented here (update: corrected link!) . There are also changes to the structure of the learning resources and discourse forum and Casey is collecting links to third-party tutorials here.
Apologies to the readers and writers of Processing Blogs for the recent downtime.
Our host machine was experiencing unusual load, and both Steve and I were away on holiday last week and didn't want to risk it going down and affecting all our other sites. I'm experimenting with Wordpress cache plugins and will probably offload the Processing Blogs feed to Feedburner, but normal service should be resumed for the time being.
Your feed reader should pick up the changes automatically, and your browser should be redirected. I've done my best not to break any permalinks - fingers crossed!
If you're only interested in my Processing stuff (previously TomC's Processing Sketchbook) then you should stay subscribed to the Processing category feed, but you might want to switch to the full feed for more variety.
Please email me or add a comment if something you're looking for is no longer working and I'll do my best to fix it - thanks!
Next up, a customised less bloggy front page template, and more content...
Nicholas Street, a recent MEng Computer Science graduate from Imperial College London, posted last week to the mysociety maps mailing list about his final year project work, TimeContours: Using isochrone visualisation to describe transport network travel cost.
His work includes a comparison with my own maps, which he says are "effective prototype implementations, but the unfamiliar unlabelled layout makes it difficult to relate to the underground". Touché! To his credit, Nicholas addresses almost all the deficiencies of my tube maps with his own software and goes significantly further in implementing the same kind of analysis for other transport networks (even including an example of using street data from my friends at OpenStreetMap).
His approach and background reading are covered in detail so the final paper will be a great resource for people working in this area in the future. I do hope he finds time to release the software for us all to use too. As well as the more traditional academic and print references, it's nice to see a hat tip to people putting their thoughts and experiments online such as myself, Rod and Oskar. Whilst a blog is no substitute for peer review and academic rigour, I strongly believe that the more of these ideas we share then the better all our work will become.
Over on the Processing.org forum, Aaron Steed is putting together some of the AI code he's written into a library, starting with examples of A* pathfinding, genetic algorithms and neural networks.
I did my own implementation of A* based on the same tutorial recently. I've put it online here since our coding styles are quite different and it might be useful to have an alternative example to work with.