« « David Carr at Medill

Guest Post: Integrating data with our journalism » »

Django app #2: Conquering forms and Google Maps API

It’s been a busy week for the programming journalists — as I’m sure you’ve seen. Congrats to all, especially the dev team at The New York Times, who just released the newest version of the Congress API, with plenty more robust features to play with, as well as my recent Data Delver interviewee Andy Boyle (who is #11 in my backlog, I’ll get to you all before NICAR, I promise!) who just released a Django app, with the help of his St. Pete colleagues.  I’m sure there’s more.  For too long, I didn’t join the party, not having a platform of my own, like Boyle’s St. Pete.  But after building one of these things, I just can’t stop seeing the opportunities. So this happened!

I’ve been working on my independent study, exploring data looking at the persistence of arts galleries in Chicago.  There’s a lot of interesting patterns that I’m both reporting for my arts class, and visualizing for the project. But what I think what would be a really helpful part of the presentation is to let people look at the parts they care about.  My independent study advisor Rich Gordon and I discussed not using Processing to make the visualization — which was our original plan — but focusing on the best tool for the job.  I’m moving the graphical part of it back into Flash.  That wasn’t enough, though. I wanted to open it up as a searchable database that will accompany this story, and the finished piece is helping me perform analysis, too. 

This quickly morphed from a quick supplement to a wrap-your-head-around-it-now-this-is-serious Django App #2!  Check it out here, download original data here, go to GitHub for code here.

Forms didn’t seem too complicated, but it was much tougher going than I imagined.  I had the most trouble wrapping my head around how to use code to approach multiple values returned by the user, you can select as many different categories of gallery as you want, and the form will return the results. Before I learned that you could search IN a list, the code was merely picking up the last value selected.  And then, what’s the point of a multiple? There were also issues with the Apache settings and using POST.

What’s funny about coding is that I’ve spent hours hashing this out, and I don’t really remember many of the specific problems at the moment.  I remember their solutions, but not what went wrong.  I just see it all as part of the process.

As an added feature, I thought maps would help people place where these different galleries were, not necessarily as an aggregate, but as you’re reading about a gallery in the detail view, it would be nice to contextualize it visually.  But this didn’t call for a scrolling interactive, Javascript-heavy piece, just the Google Static Maps API.  I’m pleased I was able to get the geocoding done through a Python script, and then use Django template tags to insert lats and longs.  It’s just the beginning of working with geographic data, but was especially meaningful because after I finished my script and ran it, I came across a sample geocoding Python script in my email that Derek Willis had sent me back in the fall.  That was the night I finally “got” what Python and Ruby were capable of.  And I was able to reproduce it on my own!

It’s that exhilarating feeling that makes all of this worthwhile, in a selfish sense.  Sure, I do it because I believe this is a valuable type of journalism, and I think it has great meaning and potential for the general population.  And in this day and age, I think it’s essential that we make the most of data, and use the technology that’s available to us — there’s so much potential!  But, in a more visceral sense, I still get a kick out of seeing something filter just right, map out just how it’s supposed to.  There’s really something special about a piece that allows people to fulfill their own curiosities in a way that matters to them, in a way that simply isn’t bound by a linear storytelling format.

I’ve been thinking about my coding experience as being a lot like stumbling around in the dark.  First, I just stood still, heightened my senses and observed my surroundings to get a sense of the environment.  Then, I started slowly feeling my way along the wall, trying simple things.  Found others who could help, became confident enough to step away from the safety of the wall, knowing that as I tripped and fell, others could help me get back up.  (On this particular project, many thanks to Ken Schwencke, Andy Boyle, Brian Boyer, Chris Amico’s blog posts, and of course, the ever-patient and responsive Derek Willis who started me on my quest.  No bias, just who happened to be online at the right time.)  I metaphorically fell way too many times.

I’ve found deployment to be the worst part, by far, and it gets a little better each time I do it.  Being that I hadn’t really heard of Apache before a week ago, well, let’s hope it improves. On the same vein, I spent two hours trying to install SQLite, adjust the PYTHONPATH etc., since I got an error that my SQLite database was unreadable.  Turns out, I still had the local path set in settings.py.  And I made the same correction last week, just forgot.  I’m coming to think that the answer is almost always the missing parentheses or slash, or something miniscule.  Occam’s razor: The simplest explanation is often the best one.

Now, I find myself running a little too fast sometimes, forgetting that I can’t really see where I’m going. Running into a wall at full speed hurts.  But it’s so much better than standing still!  And I know how to get up myself, and where I can go for a helping hand.

Back to the app, the design could use improvement, and so could the logic of some of the detail pages. In fact, a lot of that is taken from the last app.  One thing I really want to work on is fixing the output of a list of categories associated with the gallery, it yields a list separated by commas.  But there’s no “and” before the last item.  I hope to fix it soon, although whether I can puzzle that out before the end of the quarter is iffy.

There’s always something new to try, and I know I’ll tweak this further.  I suppose it’s sort of an example of agile development in a way (a technique described in practice at Medill here by the Chicago Tribune’s Brian Boyer.)  But I do think I have a much better handle on forms now, which I know has to be quite important in the CAR/Web-dev world.

If you have any feedback about the database, or happen to find any data points that stick out to you personally, I’m always interested in chatting.

« « David Carr at Medill

Guest Post: Integrating data with our journalism » »