« « We must understand our news content as data

Using Google spreadsheets as your database » »

Tools to help bring data to your journalism

Posted by on Nov 8, 2010 in Blog, data visualizations | 5 Comments

NOTE: This entry was modified on the evening of 11/9/10 to deal with typos and missing words, resulting from posting this too late the previous night.  Sleep deprivation isn’t always a good thing — although it allows one to do things more fun than sleep.  Like play with data. Note to self: Be more careful in the future.

Many of the stories we do every day, across beats, could benefit from a data component.  Luckily, a lot of great design and programming folks have created tools to make it easier to organize, clean and display data.  Helpful whether you just want to get a table up quickly, or want to spend months creating an app.  Because we’ve all got a lot to do, and spending more time on creativity is always a good thing. Let the computer deal with the boring parts.

So, here’s a round up of some tools you can use to rapidly produce data pieces without programming knowledge.  There are many others I continue to play with, and new tools are cropping up every day. The best way to get to know all of these is by taking a data set and playing around with the various options.  Each category is organized in my approximate preferential order, which should be taken with many grains of salt.

Prepping tables

  1. Tableizer – http://tableizer.journalistopia.com/ – Copy and paste cells from your Excel spreadsheet into this tool, and it’ll spit back a formatted HTML table that you can copy and paste into a CMS of your choice.  Pick the font and header background color.  Make sure your first row consists of headers, as these will be styled differently.  Combine this with the Jquery DataTables plug in listed at the bottom of this info sheet, and you’ve turned your mundane Excel table into a fancy-schmancy searchable, sortable and paginated table — a data news application in a box.
  2. TimeFlow – http://wiki.github.com/FlowingMedia/TimeFlow/ – Plot time-related data on a timeline.  Useful for looking at chronology behind a project, or ongoing story on your beat.  Helps you structure your information.  Made for reporting, not presentation.  Resulting timeline is not embeddable.  To use this tool, you have to download it on your computer, either Mac or Windows will work.
  3. Freebase Gridworks -  http://www.freebase.com/labs/gridworks – Downloadable tool to help you clean/analyze your data.  Import your data set, and it will look past minor discrepancies, like capitalized and uncapitalized letters, and group your data.  This can help you find patterns that will aid your story-finding abilities.  You can customize how it groups the data, so it isn’t aggregating information that it shouldn’t be aggregating.
  4. Mr. Data Converter – http://shancarter.com/data_converter- Enter your data in this web-based tool to convert your Excel file to other data formats.  JSON is especially useful for feeding your data to a Flash developer.  It won’t directly help your presentation, but it will convert your data into a format that’s easier for presentation tools to read.

Interactive viz – no programming

  1. Many Eyes – http://manyeyes.alphaworks.ibm.com/manyeyes/ – A fantastic tool for presentation, built by some of the best data visualizers in the business, who now work for Google.  Import your dataset, and get back an embeddable visualizaton.  You can use it sort through and analyze lengthy text, and it’ll give you a graphic-friendly word tree where users can see what ideas pop, and how those ideas are connected in a speech.  Or use it for intensity maps, where certain states are colored by the value of data (how does Calif.compare?).  Or use it for interactive bar/pie/line graphs.  No programming knowledge needed, but while you can customize colors, it can be tough to customize the genral feel of these applications.  More than suitable for a blog entry, though.
  2. Google spreadsheets – http://docs.google.com – Feel like everyone knows this baby.  Use it to structure your data into various columns and records.  Share it with your coleagues.  Work with it from a variety of different machines.  Recognize Google may be looking at your data as data, so don’t use it for your super-secret investigative project that must never see the light of day until you say so.  You can use this spreadsheet to feed a mini-version of a Web app, and pipe it into a Web page, like as an HTML table.  More on that in a post TK.
  3. Google Fusion Tables  – http://tables.googlelabs.com/Home – A lot like Google spreadsheets, but includes built-in embeddable viz tools.  My favorite has to be an intensity map.  If your data has a US state, or a country, and a value, it will shade those geo areas according to that value.  Sweet.
  4. Socrata http://www.socrata.com/try-it-free – Really helpful for organizing data and spitting out charts, that are embeddable.  Limited interactivity.  Good for analyzing complicated data, and tend to take up a large portion of your on screen presentation.
  5. Tableau Public – http://www.tableausoftware.com/public/ – Nice, relatively easy way to make charts.  You do it through an app that you download, which is PC only, as of now, I think.
  6. DISCONTINUED – Open Flash Charts – http://teethgrinder.co.uk/open-flash-chart/ – This was cool, but no longer exists.  Plus, be forewarned anytime you use a tool that involves Flash, it’s not iPhone/iPad friendly, as of now.
  7. EditGrid – http://www.editgrid.com/ – Makes simple graphs.  I found this one to be disappointing, and lack the visual punch I hoped for.
  8. DISCONTINUED – Swivel – http://www.swivel.com – This used to be another favorite, and ranked pretty high up, until one day it stopped working.  A discussion on the NICAR list confirmed it wasn’t just me.  I’ll always remember it as how I spent 10 minutes of an arts journalism presentation analyzing data of what kinds of books have been sold over time, and my professor saying, “THAT’S what the independent study is?  That’s. So. Cool.”  It is.
  9. XML/SWF charts – http://www.maani.us/xml_charts/ – Another limited and disappointing option.
  10. GunnMap – http://www.gunn.co.nz/map/ – Another limited and disappointing option.
  11. ChartGo – http://www.chartgo.com/ – Another limited and disappointing option.

Static viz

  1. Juice Analytics chart maker – http://www.juiceanalytics.com/chartchooser/ – Nice templates for making better looking graphs in Excel.  Great if you need something pretty quickly, and aren’t completely invested in interactivity.  You’ll upload this as an image.
  2. Excel – By default, the spreadsheet program makes some graphs.  Great for analysis, but you’ll want to dig into settings to customize for a better look if you want to use for professional presentation.

Use programming to make custom charts

  1. Google Charts (static) – http://code.google.com/apis/chart/ and interactive – Love, love, love.  You can use these as standalone images, or interactive pieces, on a site, or integrate it into a Web app, or anything in between.  Uses Javascript, which makes it accesssible across platforms.  You can make almost any kind of graph you can think of.  Tutorials will walk you through the different types, and you can test your ideas on the fly in the Live Chart Playground, linkable from the left of any documentation page.  More news orgs should be using these puppies early and often.
  2. Jquery (Data Tables) – http://jquery.com/; http://www.datatables.net/ – How to bring interactivity to static data-driven webpages.  Think of the various plugins as pre-made apps you can add on.  It does use Javascript, so you need to get comfortable with that.  But Javascript works  across browsers, and on non-Flash-friendly mobile devices.
  3. Lovelycharts.com – Not as robust as the other options.  I feel like you can get these sorts of results via the non-programming options, with less time investment.
« « We must understand our news content as data

Using Google spreadsheets as your database » »