« « Data Delver: Chase Davis, California Watch

“Demos, not memos”: My first Django app » »

Is a flat text file or a database right for an app?

I’ve been busily coding away on my final project for my data visualization independent study. I’m looking at data that I’ve been slowly acquiring on the life of art galleries that were in Chicago in 1990.  Back then, the Chicago Artists’ Coalition had a complete listing in a pamphlet (more like a book) they put out for artists.  It included galleries within the city, in the metro Chicago area, and even downstate.  I’m choosing to focus on ones that were within Chicago’s city limits in 1990, and I’m looking into what happened to them.  Ideally, the visualization will use the principle of a bubble chart to show how many galleries, of those 96, existed in 1990, 1995, 2000, 2005 and 2010.  You will also be able to filter by zip code, which sheds light on what neighborhood housed galleries that have lasted the longest, and perhaps more interestingly, by medium, showing how these statistics change if you look at galleries only specializing in paintings, or only in sculptures.

One part I’ve been slowly conquering is connecting my SQLite databse to the Processing application.  My current issue is closing the database, the query just runs and runs and runs and…you get the point.  I’m using a database because these counts need to be able to adjust depending on where you click, and I figure that SQL statements are the best way to filter and aggregate the various information.

But today, in my reading from Ben Fry’s Visualizing Data, which I’m approaching the end of, Fry was discussing acquiring data.  Many of the methods I’d come across before in my own research, they focused on how to hook up the database to my interface.  Fry suggested to reconsider whether you need a database, or can use programming to manipulate a flat text file.  With the way I’ve designed this application, I need to count up the number of records where a column meets a certain condition.  I was trying to do that with a count in SQL.  But I can just have the program do a loop, and count it programatically.

Perhaps using a database is a better fit when doing joins or complicated queries, or if one has millions, or even thousands, of records.  But if performing counts, and pulling records that meet certain conditions, is all that’s necessary, right now I’m thinking that a flat text file is best.

« « Data Delver: Chase Davis, California Watch

“Demos, not memos”: My first Django app » »