I’ve spent much of my “summer” internship at the LA Times (ranging from April through present) working on a campaign contributions searchable application, to help the public explore the money that’s financing various propositions in California. The first step was getting the Prop 8 application out back in late June. I’m really into the idea of templating data apps recently. Make a custom interface, but make it resuable, so it doesn’t have to stop as one application. It’s like how we already use third-party source tools as frameworks (Dipity, ManyEyes, etc.) We should expect the same, if not more, of custom solutions we create ourselves. So, the original idea for the Prop 19 app was to apply Prop 19 data to the Prop 8 code. We did that, but as is my normal mode, more features suddenly found themselves on the list. So, in the end, we have two frameworks for campaign contributions now: the simple version and the more advanced version.

One of the large advancements in the second version is that every single donation is listed separately. In Prop 8, the legacy data passed down to me had aggregate donations, which is helpful, but means users miss out on certain nuances. So, when we were able to pass each donation in separately, I believed it was key to make each one linkable and sharable. Not only does each donation have a separate link, but so does every donor, city, state, zipcode and committee. This means if people find something and want to blog about it, share it with a friend, draw a media organization’s attention to it, whatever, they have a way to point someone else to that exact spot. We don’t do enough of it in the data apps world. Web frameworks make this easier, Flash interactives do not.

Creating permanent links to records and aggregate pages of our data is essential. It’s how we combine our searchable databases with other reporting forms that provide even more context. Did you write a story about someone you know gave a lot of money to a campaign? And there’s a searchable database to go with that campaign? Don’t just tell me they donated a lot of money, show me.

Just an example of one way we can change our mindset as reporters, Web producers, developers or whatever it is we do. Find the minute detail, make it shareable. Because while you may only care about record #1157, someone else cares much more about record #1158. And both of you should be able to find that piece of the story that matters to you and use it. That’s data for personalized news, and yet another lesson I take away from my time at the LAT.

PS — If you want a random Django tip (come on, you know you do) {% ifchanged %} is my new favorite template tag. If you have a big list you’ve pulled out as a query, and it’s sorted, but you want to separate the list into subcategories, you want this. Figure out an attribute that’s going to change, and enter that info as a label, inside the {% ifchanged %} tag. It’ll show up when the code and text inside the tag changes — makes sense. This tip has made an ongoing project a lot easier, which I’ll be able to chat more about in a few weeks once it launches.

Explore other posts:

• October 30, 2011 -- Answering some FAQs about Fusion Tables
• February 15, 2010 -- Visual confections are more than mere presentation
• June 13, 2010 -- Post IRE wrap-up + shameless self-promotion
• January 19, 2010 -- Changes in the numbers of students majoring in programming and social sciences
• March 24, 2010 -- Bringing data journalism into curricula
• March 29, 2011 -- Gantt rant: Advice on agile project management

So, there’s that. First launch!

My checklist:

• Figure out what part of journalism inspires me. CHECK (Data, programming)
• Find place where I can learn about it from others, but have freedom to try out my ideas and learn from knowledgeable and patient folks. CHECK (LAT)
• Use this opportunity to learn enough about programming to create at least one app. CHECK (See first line of post)

That was a good time.

I’m planning a post on best practices for creating a searchable database application like this, with examples, and technical geekery, and all that, and I’d love your thoughts in the meantime.  Let me know if you have specific questions.

But in a moment of self-reflection, I’d just like to say this.

I was sitting in my apartment this morning, and found that a butterfly had somehow gotten in (blame a broken screen, maybe).  It sat on my windowsill, facing the outside world.  It would feel around the window, trying to find an opening.  After circling the perimeter of the window, it laid back down, almost motionless, just searching.  An hour later, the cycle repeated.  Wanting to help the poor butterfly, after watching it struggle, I tried to capture it in a small box to take it downstairs.  It was too smart for that.  Fine.  What actually worked?  Covering all the windows and opening the balcony door.  You can’t force the butterfly out, even though you’re helping, it had to find the path out on its own.

Why do I bring this up?  Am I slowly going insane after spending the day with Dive Into Python (rec’d)?  Perhaps. But I have a point.

Let’s take the common journalist, yearing to know more about data-driven apps.  (I can’t be the only one, I just can’t!)  I search for the answers, voraciously read, alternate between laying still while observing the space and wandering around the perimeter, feeling it out, trying to dip my toe in the water.  I jump into the pool, head-first, realize it’s not working.  Can’t do it on my own.  Many people who’ve already made the journey try to help.  But they just can’t just give me the knowledge, can’t just take me to my goal, can’t just make it happen.  They must lead me there, give me the same resources they used.  Recommend the books, the sites. But in the end, I’ve got to find my own way out of the house, the prison of not knowing enough coding basics to begin.  Pushes in the right direction, guiding me away from those false starts, this is essential.

I’d just like to take a moment to thank the experts, who guide me away from the false starts, who don’t give up when I keep turning the wrong way.  In particular, in terms of the programming skills I’m learning, it’s Ben Welsh (who I happen to have been working most closely with), who exercises that kind of patience.  Who never tells me not to fly into that closed window, because while it was closed for him, I might discover a way to get out.  Of course, I don’t, and he’s no less interested in helping me get back on course. Thanks!

Still a long way to go.  So, what’s changed?  Now, I’ve gotten outside.  I made my first app.  When someone asks me what steps make up a project, how long it’ll take, I can give a legitimate answer, or know how to find one.

I still remember wondering if I could ever build an application.  And Derek said to me, “There’s only one way to find out.”  Derek, who I used to bug on a daily basis about this stuff.  Derek, who I haven’t asked a Python/Django question of since March.  I’m the LAT’s problem now!

Well, as Derek said, find out we did.  Time to push it further, I say.

There’s a whole world to explore now that I’ve escaped being stuck inside.  LAT folks, prepare yourself, the questions you’ve seen so far are only the beginning.

Explore other posts:

• January 4, 2010 -- My new perspective on math – it’s a journalistic tool!
• July 30, 2011 -- What’s it like looking for a programmer-journalist job?
• March 29, 2010 -- Using Javascript for interactive Google charts
• March 26, 2011 -- My (quickly formed) vision for a journ-prog curriculum
• September 28, 2011 -- Journonerd Crisis: Where am I, and where am I going? (ONA Wrapup)
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today

Visualizing through graphics

The Flash component allows you to search by art medium accepted, or by neighborhood.  The neighborhoods are determined by groups of zip codes, and the ten art medium categories are made of various combinations of the more than 60 categories itemized in the Chicago Artists’ Coalition’s list of galleries.  One important thing I’ve learned about data reporting is that you have to use journalistic discretion to pare down the options.  60-something categories of media can be extremely overwhelming.  If you fail to use discretion when creating a project, you’ve failed the user.  I used Tuftian principles for the graph itself, keeping it as simple as possible.  I liked using the idea of duplicates to make up the units of a bar graph.  I went with houses to symbolize the gallery buildings.  My first instinct was to use paintbrushes, or paint palletes.  But that doesn’t work when you are sorting for the decline of these galleries that sold wearable art. The Monopoly-style house was the best generic choice.  I went with the blue accent color since it was calm and unobtrusive, but more visually interesting than a solid black and white graph.  I’ve been doing a lot of reading about the power of an accent color, and sticking mostly with the neutral to pack a punch.  Other tweaks I made as I went along included adjusting the size of the ticks on the y axis.  I found myself thinking more about interface design than usual, when you have a certain filter on, the background color on that button should be different to indicate what variables you are looking at.

And I learned something about myself.  When I set out, this type of piece is what I saw as the definition of a visualization.  Now, I see data-driven applications, especially those allowing for you to dig deep using the power of frameworks, as a form of visualization. One isn’t necessarily better than the other, in fact, I think they play off of each other.  But I’m not a visual person.  It’s fun to play with, but I know designers who could have packed more of a visual punch with this piece.  But by keeping it simple, I’m hoping it works.

Searchable databases are visualizations, too

You can read most of my thoughts on how this came out here, and I’ve made a few small tweaks since then.  Most notable is dealing with the list of media accepted on the detail page.  I was trying to form a sentence with the media that were accepted, but was having trouble figuring out how to display “and” before the second-to-last template tag that was actually displayed.  I solved the problem by turning the accepted media list into a list of bullet points.  I think it reads better this way, too.  I like the idea of being able to get an overview of sorting the data through the Flash visualization, and then digging as deep as you want with the searchable database.

Not providing current gallery information

Going back to the Flash visualization, throughout the quarter, my independent study adviser Rich Gordon and I debated the merit of putting the number of current galleries into the visualization.  I eventually opted not to.  That’s because this whole project is about persistence of art galleries — what happened to the 96 that existed in 1990?  Introducing the number of current galleries is a completely different data set.  I tried making it another node on the graph, but it made everything else seem tiny.  Even listing the number of current galleries just seemed out of place.  I believe the lesson here is to stick to your data set.  Comparing the galleries that persisted to total number of galleries is a valid project, but a different project — that was my ultimate conclusion.

Flash as a viz tool

Some other challenges I encountered included proper alignment of the various house icons (thanks to Flash’s grid feature for help on that).  Knowing when to use the right tool was an interesting battle.  At first, I was trying to get Flash to create the detailed views that Django is ultimately better suited for.  By combining Flash and Django, I think I came up with a significantly better combination.  Even if someone doesn’t explicitly go into design, I would heartily recommend they read Tufte and try experiments in Processing, Flash and Django.  It’ll all help in the long run.

Looking toward the future

There was a time, not that many months ago, when I prided myself on the fact that I knew enough about programming to communicate with coders.  Now, I just do the programming myself — or at least, as much as I can.  I’ve determined that visual design isn’t my forte, but I understand enough about it that I’ll be able to integrate smoothly with an art director or Web designer.  I even understand enough about Flash that I’ll be able to collaborate closely with front-end interactive designers.  But I wouldn’t know any of this if I hadn’t tried a lot of new things. I’ve applied my journalistic curiosity, that I typically use for subject matter, to new ways to tell stories, and tools with which I can bring those stories to life.  It is for that adventure that I embarked on this journey.  As I said after my Washington quarter, I’ve learned a lot, and I’ve also learned just how little I know. It will be an ongoing quest. But how fun and educational it’s been to ramp up my skills in these past ten weeks.

The power of community

As I’ve been pursuing the independent study, one of my other goals has been to blog regularly.  By that, I mean more often than once a month, which has been my typical pattern.  And through the excitement I’ve gotten from my exploration of programming journalism and visualizations, I’ve been able to do that.  I’ve promised myself it won’t end at this point, there’s so much more to explore.

But if you’re reading this, I want to thank you. Your comments on the site, via email, or on Twitter have enhanced my learning process.  You may not be receiving tuition money, but whether you helped me troubleshoot my Django app, discussed the intracies of Flash, pointed me to a great GitHub tutorial or just chatted with me about where I might fit in the grand scheme of things, I couldn’t have done it without you.  The greatest asset of the CAR community, as I see it, is the community itself.  I look forward to keeping in touch for years to come.

It’s for that reason that I couldn’t be more excited about NICAR later this week.  (Less than 72 hours till I leave!)  I can’t wait to meet you all in person, to hear even more inspiring ideas that I can bring to my first “real job” (not sure where that is yet, but I’ll keep you posted).  Heck, I’m even giving a five-minute lightning talk on some cool tricks I’ve been playing with in Javascript.  It’s a beautiful thing when an almost-graduated student’s knowledge is valued that much.  But from what I’ve seen, I shouldn’t be surprised.  It’s not about a judgment of age or experience, it’s about sharing something with each other.  That’s what makes NICAR, this quarter and journalism in general so special.  As it says on my high school class ring, knowledge is power.  I’m ecstatic that we all will continue that journey together for years to come.

Related posts you might enjoy:

• March 3, 2010 -- Persistence of Chicago Art Galleries
• January 10, 2010 -- Which states have been hit hardest by unemployment in the last decade?
• February 28, 2010 -- Django app #2: Conquering forms and Google Maps API
• January 18, 2010 -- Committing fact errors in visualizations
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• February 15, 2010 -- Visual confections are more than mere presentation

Gallery owners: Flexibility necessary for survival

Chicago’s local art gallery community is a tight-knit group.  Ask them about the industry’s history. They’ll tell you this city was once seen as a mecca for new, up-and-coming art galleries – but that was in the mid- and late-1980s.  And things have changed since then.

There was a time when dealers at these personalized institutions sold everything from paintings to kaleidoscopes to cards – all to a stream of clients with a variety of interests from across the globe. The pieces they sold gave customers a respite from the large-scale department stores that had emerged in earlier decades, where so many items were mass-produced.  Art collectors knew they could get something special at one of the at least 96 galleries that existed in Chicago as of 1990.

In fact, the Chicago Artists’ Coalition put together a list of these galleries that were within the city limits, as well as others in the suburbs and further downstate. It served as a resource for their members to learn where they could sell their work.

Fast-forward to 2010, and some gallery owners will tell you the heyday of the 1980s has vanished, and Chicago isn’t able to support galleries the way it once did.   Others don’t bemoan the health of the gallery industry.  They say it’s manageable, if you’re smart about your expectations for your business.

Deven Golden, director of the Compassrose Gallery in 1990, said, “We used to say we were an international gallery that just called Chicago home, but that dried up at the end of the ‘80s.”

Golden, who managed the River North gallery that closed in 1992 after founder Jim Rose’s death, has since moved to New York to pursue his career as a gallery owner.

The persistence of these galleries, their ability to survive, is one way to measure the health of the visual art industry in Chicago.

The Medill News Service followed up with representatives from the 96 galleries listed in the city as of 1990, to find out their fate.  Analysis of a database created from this information revealed that six other gallery owners had similar thoughts to Golden, and they have moved their galleries out of state since 1990.  And 34 of the 96 galleries have relocated within Illinois, often remaining in Chicago.

49 galleries from the original list are now closed.  That’s just over half of the total.

To survive the change in the economic climate in the past 20 years, some gallery owners say they have expanded their inventory into other areas. While they still consider their businesses galleries, they have other interests.

Karen McCauley is the manager of Gallery Genesis, located on the South Side.  It’s one of seven galleries to stay in its same location since 1990.  McCauley credits the business’ success to a shift in focus.  “We’re more of a church goods and religious supply store now, than anything else.  But we are still a gallery.  We keep artwork on display upstairs, and we do sell it.”

She said the gallery was successful with its original focus on religious art until the early 2000s, and then it just wasn’t bringing in enough money in its current form.

“We’re in a niche that’s very small and specialized, it’s definitely hard,” McCauley said.

Gallery 1633, located on the Northwest side, is one of the nine galleries from the 1990 list that has closed in the past five years.  It closed in 2007 when co-founder, and then-owner, Montana Morrison decided to move it to Red Bluff, Calif.  Bill Dixon, husband of the late Morrison who died in late 2009, said that the gallery was affected not just by the artistic market in the area, but by the neighborhood as a whole.

“It just wasn’t like it used to be, wasn’t the same experience to live or work there,“ said Dixon.  “It was overcrowded, generally.  It wasn’t a comfortable place to shop, or to live.”

Others thought the prevailing issue was the lack of visibility for the gallery community in Chicago by the public at large.  And that’s a problem for a business in an industry where so much relies on word of mouth.

River North’s Center for Contemporary Art closed in 1996.  According to Cheryl Pelavin, who worked at the gallery in the 1990’s, the reason was clear: “Lack of prestige,” she said simply.

For many gallery owners, these issues meant that their love of art dealing often took them away from the city they called home, as they found the market increasingly difficult throughout the 1990’s and 2000’s.

Golden of the Compassrose Gallery now owns his own gallery, Deven Golden Fine Arts, in New York.

“In the ‘90s, we sold 60 percent of our art to people outside of Chicago, “ he said of his work at Compassrose.  “I don’t think we were alone.”

Golden said that for many international clients, it was often easier to get to New York from Europe.  The Big Apple also created a certain experience for these collectors, which Chicago simply didn’t have, he said.

“Collectors like to go to New York,” said Golden.  “When people were starting their collections, they’d go to Chicago, and it’d be fine.  But when people started to buy enough, suddenly it would matter and click, and they ‘d move on.”

At the end of the 1980’s , the gallery scene was on its way up,  Golden said.  At that time, he thought it might meet the success and prevalence that theater now enjoys in Chicago.  But that would have required more substantial backing from the city.  “We never got to the critical mass that we needed.  It just didn’t happen,” he said.  “The city had a lot of trouble forming its own identity as an art community.”

And Golden said he wishes that weren’t true.  “I could’ve stayed in Chicago.  But I liked the gallery thing. I thought to myself, ‘I want to have my own, and in Chicago, it’s just very difficult to do.’”

The declining climate for galleries isn’t specific to Chicago, though. Golden owned a gallery on Manhattan’s 57^th Street for five years, and in the early 2000’s, he lost his backer, and had to close that gallery.

He’s now opened a new one, but he sees it as more of a side business that supplements his work at Artsystems, where he designs software for art dealers.

Yet others have made it work in Chicago. One such gallery owner is Cheryl Pertl, whose A.E.S Gallery has persisted in Chicago since 1990.   The business has moved three times, but has always stayed within the city limits.  “We go where the customers are,” she said.  She added the key is being flexible as the market shifts.  While she previously focused on mediums, such as painting and sculpture, now she carries works across media but only covering one topic: McDonalds. She said that attracts a specialized, and dependable, client base.

Pertl now also owns the Ogilvie-Pertl Gallery, which opened in 2000 in the River East Art Center.  As opposed to the modern incarnation of A.E.S., it has a broader specialization.  Her tip for success is to recognize that treating a gallery as a sole source of income may not be feasible.  “I make my own art, too.  If you think you can just exist on the money you make from a gallery in Chicago, well, it’s not that you can’t, it’s just extremely difficult.”

Related posts you might enjoy:

• March 8, 2010 -- Piece de resistance: Data viz wrapup
• January 10, 2010 -- Which states have been hit hardest by unemployment in the last decade?
• February 28, 2010 -- Django app #2: Conquering forms and Google Maps API
• January 18, 2010 -- Committing fact errors in visualizations
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• March 2, 2010 -- Guest Post: Integrating data with our journalism

It’s a large part of people’s lives, and deserves further study.  That’s the motivation behind this interactive graph I cooked up using data from the Bureau of Transportation Statistics.  It’s interesting to see just how much more people are using cars to cross over from the United States to Canada or Mexico, and how that compares to traffic from buses, trains and pedestrians.  I was actually quite surprised to see just how prevalent walking is in some states, second most in overall traffic to passenger vehicles.

Here, we have an visual exploration of how total traffic across the U.S. border has changed since 1996, in terms of how much each state and each mode of transportation contributes to the overall amount of those coming in and out of the U.S. on land.  Air travel was not included in the dataset, and rightfully so, do you count the planes that are passing over the border, or just ones that land in Canada or Mexico?  It seems as if it adds a lot of unnecessary issues.

Also, the data only goes up through July 2009, so when it looks like there is a drop from the previous year, it may be because a full 12 months aren’t recorded.  This seems to be a common issue in data analysis that I’ve come across in other projects too.  I don’t think you should leave it out, because that’s denying the user information, but it does merit an explanation.  I welcome feedback on this, one of my more ambitious projects to date.  (More analysis and personal reflections after the jump.)

I thought the best way to evaluate this data set was by enabling the user to both see the big picture and make comparisons across states or modes of transportation.  (Full disclosure: This data set was from my final project for my digital frameworks class, but I didn’t get to complete it as thoroughly as I wanted.)

I still haven’t done as much analysis as I would like, but a new quarter calls.  (And if you’re wondering what the distraction was, it was designing and implementing the web site for a quarter-long team reporting project on religion and the environment, as well as data crunching on a piece Medill did with the Tribune looking at former staffers from the Hill now lobbying for organizations with health care interests.)

On the technical side, this was an adventure deeper into ActionScript than I had ever gone before, and much of it was based on this tutorial from Flowing Data.  I was exploring Adobe’s Flex program for the first time, and it was both challenging and rewarding.  Using Flare, a library created by the UC Berkeley Visualization Lab, helped me to take a lot of shortcuts, but at the same time, I found you really have to get inside someone else’s head to understand where different code is stored.  This emphasized the importance to me of keeping my own code as clean as possible — something I’ve heard often, but rarely experienced from the other side.  Flare was actually very well-documented in the comments, it was just an issue of searching through many different classes for the right line.  I suppose it’ll get easier with experience.

I wonder about this lesson as it applies to journalism. So often, I think of reporting as an individual job, but if we share work we have done in the past, we could be more efficient and provide even more information to the public.  I guess it’s an argument to stop bragging about how my chicken-scratch handwriting gives me the skills to write in investigative code no one can read, and think more about how to better facilitate the exchange of information with colleagues.  It’s the same rationale as to why interactive projects come out better when done as part of a team.  Sure, I can report, analyze, design and produce packages myself, but it would come out so much better if I were working on a specialized part of it with others who had expertise in different components.

My current dilemma is discovering exactly what my specialty is.  That’s why I’m embarking on a data visualization independent study on Monday, a major component of my final quarter at Northwestern.  The journalism industry and I share this commonality: We must work to determine where is it we fit in this changing world.  It’s not a question of if there’s a place for us, but a question of how best to discover and focus on our greatest strengths.

Explore other posts:

• January 31, 2010 -- Data Delver: Tyson Evans, NY Times Interface Engineer
• November 22, 2009 -- Printies and webbies — we’re all journalists!
• May 24, 2011 -- Oh, hi, blog readers — it’s been a while!
• February 15, 2010 -- Visualizing networking: When it doesn’t work
• December 1, 2010 -- “I do not believe programming replaces the story” No, no, no!!!
• January 2, 2012 -- The word of 2012 — Moderation