Tuesday, 11 February 2014

OpenRefine and Custard

I got my first taste of Google Refine which is now OpenRefine - a pretty classy little app for wrangling datasets. There are some elements of Excel-like-ness to the tools, but they are superficial. OpenRefine is all about massaging data, bringing in fields, generating columns based on others and so forth. There is quite a reliance on the GREL scripting language to achieve tasks - it is quite intuitive but the help for it was frustrating in that searching for what seemed like simple terms didnt come up with anything.




These 3 videos are from the sites homepage that show off the features of the software. I like the concept of using the toolset to draw in data from online data sources. In the training we brought in latitude longitude info from Google Maps and data on police stations from the NSW records site (http://search.records.nsw.gov.au/usage) which feels very powerful. Some skill is needed to work with extracting the actual data needed from the various data sites and their various data structures.

The research implications of a little tool like this could be huge - helping get the right data to the right people to extract understanding from it all. It was fun at the training session yesterday here at UNSW BE seeing the Urban research guys exploring how they could bring in data to many of their projects. Seeing how they might leverage their involvement in the AURIN project to extend their software toolsets.

We also had a little self-directed peek at FusionTables and using map data - another world of data visualisation right there!

---

Well, if you made it this far through this post that can mean only one thing - you are intrigued about the custard. When I was walking along the other day this idea popped into my head ... Protective Custard ...


No comments: