I’ll just put it this way.  Andy, you’ve been an inspiration.  Watching your work while I was a student at Medill, and how much you enjoyed it, I knew some day I could do anything, if I could just set my mind to it, and find supportive mentors.

It’s a parallel thing.  Andy gets Matt Waite and Jeremy Bowers at the Times of the Southeast (St. Pete), I get Ben Welsh and Ken Schwencke at the Times of the Southwest (LA).  Match good mentors with journalistic enthusiasm and obsession, and you’ll get somewhere!

We both get to bring passion, skill and journalistic knowhow to the table.  We bug the people with the tech knowledge until we have a moment like this.   Maybe someday I’ll report in the field and code like Andy does.  But for now, my reporting consists of investigating the nuances of the still-large ship that is the LA Times, and looking at how it can be even better.  And at this point in time, I wouldn’t change it for the world.

It seems appropriate to post this interview with Andy now (Yes, I’m making excuses for delaying this for months and months….).  I chatted with him in March, just after the launch of his first Django app – MyLawmaker.  And I just launched my first one about two weeks ago.

I started these Data Delver interviews knowing no one in the field other than Derek Willis.  (And if you only got to know one person, he’s as good as it gets.)  And now, I feel like I know so many more.  And for some reason, I get to be a part of their ranks each day.  It’s a privilege, an honor, and an adrenaline rush like no other.

I’ll let Andy take it away in his own words now.

This transcript of my interview with Boyle is a part of my continuing series I’m calling “Data Delvers,” where I pass on transcripts, summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Extended transcript

Why is having an understanding of data important?
When we have meetings to discuss story ideas with the group I work with we mainly focus on breaking news on the web.  When somebody says something, it’s really nice to be able to go, “Oh, I know how they keep those records so I know exactly how you can ask questions to find out how many total XXX has happened this year.”  It really helps to have a background, it gives you a chance to dig deeper and get more context.

Is St. Pete working to get other reporters to get more up to speed on this?
I think there’s quite a few reporters who do have this type of skill or do have the basics.  They can use Excel; they can use Microsoft Access.  Actually, there’s a reporter who did a really awesome story on leaky underground storage tanks and where they are located.  My compadre, Darla Cameron, does a lot of GIS stuff, so she’s also a data nerd, like me. She was able to help him with that. A lot of people use databases on a pretty average basis and have some CAR training, but not as much huge stuff like the Wetlands project that the St. Pete Times did with Matt Waite and Craig Pittman a few years ago.  But things on a smaller scale are happening.

Going back to the MyLawmaker project, what was the genesis of that?
The genesis is that on Monday the state legislature starts so we print this thing every year in our Perspectives section which runs every Sunday.  We print opinion pieces and stuff like that in a section called ‘For a better Florida’.  One of our politics editors, Amy Hollyfield,  had sent an email out to one of the higher-ups  saying that there’s this project from the New York Times called Represent and another one at Oregon Live –  Your Government, where you type in your address and it shows you your state lawmakers, congressmen, city lawmakers, etc.  She asked if it would be possible for us to set up something to help people find their state representatives and state senators from their address?  This somehow made it to me and I go, “Yeah, yeah, that can be done; it’s possible.”  At that point I had no clue in my mind of how we were going to do that, but I knew it was possible because obviously someone else could do it. So we had the shape files; Darla already had the information.  Somehow we could write a program that would geocode your address and would ask what you wanted to see, and it would print out what you wanted.

So that was basically last Thursday and it is now Friday and we are ready to launch.  Thursday, about 3:30 in the afternoon, it was like, “Hey guys, do you think you can do this?”  I was taught a long time ago that if you think you can, try it. This is something that I really wanted to do.  It was an opportunity to try this stuff   I’ve been working on.  I’ve been working on some Django and Python with Matt Waite and Jeremy Bowers, so it was nice to spearhead a project and have ownership on something and work with a group of awesome people like Darla and Lee.  I’m really jazzed.

So how did you start getting into Django…just by being around Matt and Jeremy?
Yeah, that’s totally it.  He’ll [Matt Waite] browbeat you into believing what he believes.  When I went to the Indianapolis NICAR conference he had a thing, a bootcamp, for three or four of the days on frameworks, so that was Django and the basics of it.  And I remember when he showed it,  I thought, “Man, that’s way easier than making hundreds of  individual web pages and then having to edit the HTML in each one. “ I remember being a kid and having an AOL members’ web page and how horrible it was to have to update every single HTML file and I thought “Wow!  This is something that automatically creates this stuff.   That is sweet.   I need to learn this.” And that was in March of last year.

And then after that it was just me attempting to learn stuff and breaking everything and being really afraid that I’d destroy my computer and not becoming afraid of the terminal. It was a very long process.  There were a couple of projects over the summer that I worked on.  One was FCAT, which is an aptitude test given to students in public schools in Florida. We were able to make a searchable database for that.  I wrote the models on that, and then they did the rest of the heavy lifting.  I guess that was the first project I had any sort of help with.  Then we later had a project for high school sports, called Home Team, which is totally awesome.  I was able to see the inner workings  and it was really cool to see the process of how they were developing it and how it was being set up.  I wanted to build something, so when the opportunity came along for MyLawmaker, I jumped at the chance to do it.

So what did you end up using for the geocoding?
We used Google.    There’s a limit of 2,500 times per IP in a 24 hour period which is an issue if you hope to get a lot of hits on  your website.   So what we do is we actually pass it off to JavaScript on the client side, so the server’s IP address comes from the user’s computer. Not that we expect 2,500 hits a day, but we have no clue as to how many hits it will get.   But we thought it’d be nice just in case.  We don’t want people trying to use it and it’s broken.

Have you had requests from the community for anything like this?  It doesn’t have to be specifically MyLawmaker, but just more interactive stuff?
I think a lot of people really like it.  When our FCAT project launched in the summer, it got a ton of use.   It still kind of does.  A lot of those projects that are usually attached to stories initially get a lot of hits.  With a project like this once we told some of our colleagues about it, they were like  “Wow, that’s a great idea.” With the state senate website, you can only search by zip code.   Then it will tell you the districts that are in that zip code and you have to click on that and see if you are located on a little map of the district, so ours is actually a little better than part of the state legislature’s.  But I guess that’s the goal.  We want to make sure that people can find the information and find whom to call if they have an issue.   A lot of people were quite surprised…they didn’t believe who their state legislators were.  Said, “That can’t be true.”  But it was, and they wouldn’t have known otherwise.

Why do you see this type of work as important?
It’s interactive.  When you pick up a newspaper, you read a story and it kind of ends there.  What’s really cool about this is it helps you find out broader information.  You can go searching a given database online and become more aware of your community. That’s part of what we do, as journalists, and as news organizations, we try to inform the public.  To do it in a way that’s automated, that doesn’t require our constant supervision or our constant writing of stories, helps because it frees up time for us to do other stuff.  It still gives people information and more context when it comes to stories.  I believe that a well-informed populace is much better than an ill-informed populace.  It’s also really cool and lots of fun to do.

Do you enjoy living in both worlds – the reporting and the developing aspects?  What are the advantages and disadvantages?
Yes, I do.  I guess time is a disadvantage in everything we do.  Part of it is because of the reporting, I can come up with projects that we can do.  I think if I was only a code monkey and I was never out on the streets, I wouldn’t find out what data is available or what data we can build on our own. It also helps you keep track of the news.  Like this is being built in response to the fact that the state legislature is meeting on Monday to start a new session. If I was just a reporter, I wouldn’t be aware of different ways to think and inform the public.  And if I was just a developer, I would be a little blind to some of the opportunities that there are to inform the public, and make cool stuff.

Any other advice?
Something I wish I’d been told when I was younger is failure happens, especially with this sort of stuff, so you really need to get used to it, and we all fail.  Your failure is a failure, unless you learn.  Then, your failure becomes a win.  If you’re always trying to learn stuff, as a journalist, if you’re always working to develop different skill sets, whether it’s Web development, normal computer-assisted reporting, GIS, narrative storytelling, it all helps you, it all helps the other things.  Whatever new skill you learn in journalism will add to your big palette of rocking.

Would you recommend all or more reporters get into the coding side of things?
It doesn’t hurt.  It doesn’t hurt to understand the basics of data.  If you’re a city hall reporter, it helps to know basic Excel, it helps to know Access. There’s a ton of cool stuff you can do because of it.  If you are a feature writer, and you write longer narratives, and stuff like that, you can still find stories through databases, you can still find extra people to interview through those methods.  It helps in every aspect I think, just to have a little bit.  Whereas I have made the intense plunge into being a total nerd when it comes to this stuff, if that’s what you want to do, that’s what you want to do.  I don’t think that should be a requirement, but I think you should have at least the bare bones basics of what’s going on, because knowledge is power, and knowing is half the battle.

Related posts you might enjoy:

• April 11, 2010 -- Data Delver: Paul Monies, The Oklahoman
• April 5, 2010 -- Women with coding skills are no Passover miracle
• April 5, 2010 -- Data Delver: Phil Meyer
• March 28, 2010 -- Data Delver: Mark Schaver, Louisville Courier
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica

One of the areas in journalism most ripe for data work, as I see it, is enterprise stories.  At papers with small CAR staffs, there’s often a serious strain on the time of CAR reporters and editors seeking to provide context and fodder for long-term projects, often investigative, and enhance shorter one-off daily pieces, often breaking news.  Spend too much time on one area, you’re neglecting the other.  At the Seattle Times, one editor is called the Data Enterprise Editor, and her time is largely based in project work, that breaks out of the daily story routine.  She works to include interactivity on the website, from searchable databases to Google maps.  And she works with a group of reporters focusing on suburban areas, that are too often undercovered.

It’s her job to organize and foster collaboration across the newsroom, to create the best data-based stories and projects possible.  The woman who holds this title?  The Seattle Times’ Cheryl Phillips.

This interview with Phillips is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes, transcripts and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Bolding within Phillips’ answers denote some of the quotes I found most interesting.

Please describe your role as Data Enterprise Editor.

The data enterprise work is basically me and a small team, two other people. We focus on computer-assisted reporting, and then try to take that CAR work a step further and do some interactivity online with it, like a searchable database, whether it’s a Flash piece, interactive map, or database. The stories vary, some are longer projects, some are, “Let’s just create this Google map for a feature piece.” That has nothing to do with investigative journalism, but it creates a more interactive environment for the paper. That takes a big chunk of my time. The other thing that takes an almost equal amount of time would be that I’m an editor for a team of three reporters that cover suburban communities. We try to make them pretty mobile, so they’re out often, but the idea is that they do enterprise. It’s not like a daily suburban story. It’s like, outside the city, what’s going on, what are the issues of note. One of my reporters is doing a big piece about all these revised floodmaps that are coming out, and what the impact is going to be on these rural communities.

When did you start?

I’ve been doing this job a year and a half, since Sept. ’08.

How long has the paper been doing suburban reporting?

We used to have an East Side bureau, but then, when we had cutbacks, we eliminated that bureau. We also had another bureau covering the suburbs to the north. So, this is an effort to make sure that we don’t forget about those suburban communities which still want to be covered, but we can’t cover every city council meeting, so we cover important news that matters. We can give a sense of place for our readers by telling these stories.

Can you tell me about your career path to the Seattle Times?

I’ve been all over the place. I started in the ’80s back in Texas, then I went to Montana, which is my home state, and I worked for a small Gannett paper there [Great Falls Tribune]. I was on loan to USA Today for about half a year. Got back in ‘95. Then I went to the Detroit News and was a CAR projects editor there. I went from there to USA Today, and was a database editor. You probably heard the story about how I left USA Today. I was fired for touching a piece of artwork, along with two other people. I did a project for Dateline on drunken driving, and then I was hired by the Seattle Times.

Can you pinpoint any marked differences between broadcast investigative work for Dateline vs. print reporting?

It was stunning to me, the difference. I hadn’t truly recognized it. I did a lot of work for this Dateline project special. The editor from Dateline did a lot of work, too. They did a lot of filming focusing on one woman who was hit by a drunken driver, and her recovery. I spent months analyzing data from four different suburban metropolitan areas, looking at sentences for manslaughter where someone had also received a DUI for vehicular manslaughter when they were sober. The result we found was that if you also had a DUI, you got the lighter sentence on average, so you got penalized more for being sober than you did for being drunk, which is kind of stunning, and it boiled down to two sentences on air. I could have written an entire newspaper article about that, but it was important for them. They were willing to spend the time to invest in my work, but it was just a piece of this visual narrative, so it was very different.

What is it that drew you to CAR in the first place?

I just felt like there were stories that were going untold, that you could tell if you understood what was happening in bigger swaths of information. You could use these technologies and tools and become a better reporter. One of my very first projects was really simple, it wasn’t a big chunk of data. I used a Lotus spreadsheet to analyze the partnership agreement for the Texas Rangers, back when George W. Bush was one of the owners of the team, to figure out how much he would get if he sold the team, and how much everyone else would get. It was a formula, so I had to use a spreadsheet to do it. It was the first time I’d been exposed to that. I had a CPA friend who helped me with it. And I was like “Oh, wow, I can actually do this,” and it was kind of amazing. It opened a new world for me and it kind of went on from there.

What’s your role in IRE?

I’m chairman of the board at IRE, and I’m a past president.

What’s fueled your involvement in that organization?

I wouldn’t have learned anything had it not been for IRE. I started going to conferences in the mid-‘90s, and I would have gone earlier if I had known about the organization. My first conference was when I was in Montana, and I was on loan to USA Today, and my paper from Montana didn’t really have the money to send me to a conference, and the other database editors at USA Today basically said, “Well, yeah, I’ll share a room with you. My point is, I had no money and a whole slew of people took care of me, from sharing rooms to not letting me buy meals, and it was really stunning. So, I ended up rooming with someone from USA Today, and literally, no one would let me buy anything . It was just great, and I learned a ton, and I saw what was possible.

I went back to my small Montana paper and started to incorporate that into my work, and turned out some pretty good stories that I couldn’t have done otherwise. So, my involvement in IRE has come because I wanted to make sure that other people could have that same experience, and produce some great journalism based on what they learned.

As far as the future of CAR, do you see this as a skill more reporters will be required to have? Do you see it more as the role of a CAR specialist?

There will always be a role for a CAR specialist, now more than ever, really. I hope more reporters will gain basic computer-assisted reporting skills, that they will all know how to use a spreadsheet, they will know how to calculate percentage change, basic stuff, especially as we try to develop searchable applications. But when it comes to dealing with really complex data sets, you’ve got to spend some real time in that world. Because, if you don’t use these skills often, then you lose that. I’m an example of that. It’s been a few years since I even used ArcView mapping programs, I’m pretty slow and clunky now and it’s because I haven’t had a need to do that, to use that map expertise. So I go to my CAR specialist, and I have him do it, because unless I’m going to be doing it all the time, I need him to make sure it doesn’t get screwed up.

I have reporters who don’t do it all the time, they’re not CAR specialists, but they do have computer-assisted reporting skills. I think the balance is that they do the work, which is great, and I want them to keep doing that work, but it gets run by a specialist or by me, to make sure that somebody checks that analysis, just to be doubly sure. Anyway, we always check, but it’s especially important when it’s someone who doesn’t use high-level CAR on an everyday basis.

How do you decide which projects are worthy of interactive and/or searchable database treatment?

I think it depends on a couple of things. It’s similar, I think, to how the graphics department thinks about the graphics that they do for the paper. Do they aid the reader in a better or deeper understanding of the story? That’s one piece of it. Another piece of it is: Would the reader be interested in more information? On the Web site, in a text file or just in the paper? A lot of people are often interested in diving down into the numbers, that’s something which I think can help with greater transparency. If it’s one where you think someone would be interested in those numbers, or it would tell them something useful, create a database for it. If it’s not, if you don’t think people are going to go to it, if they’re not going to be interested, or if it doesn’t illuminate in some way, then we don’t do it.

How is this work broken down at the Seattle Times?

Another part of the Data Enterprise piece of my job is that I work across departments. It may be that the business editor is having a reporter working on a story about home values. So, she’ll come to me and say, “Hey, do this with us. We can create something online,” and she’ll say something about what they’re trying to achieve. Then, I’ll go to the online department, and I’ll say “What resources do we have if we create this searchable interface? Can you do some design with it, or give it some extra treatment?” It’s sort of a collaborative conversation. Also, depending on the level of time involved, I think about if it would take our CAR specialist, or me, x amount of time, because we also have this project coming. It’s really a mix. We’re very collaborative, we talk about everything.

Do you try to bring all the departments in at the outset of a project?

Yes. Pretty much. In ’09, we did an interactive Flash map, it was an environmental graphic. We won the James V. Risser Prize for environmental journalism, Justin Mayo was the CAR specialist on it. We did this really nice thing that took quite a bit of time, it was very complex. We had brought in everyone from the beginning. The photographer was there, two reporters, Justin, and another reporter. And we had the Flash guy involved from the very beginning. We took a look at logging permits, and kind of showed how there was flood damage in particular areas that had been logged heavily. We ended up using these maps, we were creating these maps for our reporting use, to say, “Oh, look, here’s this hazard zone.” We found ourselves using it so much as a reporting tool that it dawned on us, in part because we had the Flash guy involved from the beginning, that we really needed to put this online. If we use it to help us understand a pretty complex subject, then it really would help the reader.

Speaking of readers, what feedback have you gotten from the community to this sort of work?

I don’t get a lot of phone calls, or things like that. They respond to our stories saying, “Thanks for getting this information,” or “Gee, I’m really glad I can look this up.” That’s really about the extent of it. I would say more feedback comes in the way of traffic, and our searchable databases get some pretty good traffic. Even something simple. We did election results, where we created a quick searchable database for the elections, and that got an incredible amount of traffic. You could go to the various counties, and see what those counties were doing, and people seemed to like having it all brought together in one spot.

Related posts you might enjoy:

• January 11, 2010 -- Importance of combining data analysis with context (reflections on readings from week two)
• April 11, 2010 -- Data Delver: Paul Monies, The Oklahoman
• March 29, 2010 -- Using Javascript for interactive Google charts
• March 24, 2010 -- Bringing data journalism into curricula
• March 18, 2010 -- My next move: LA Times!
• February 21, 2010 -- Is a flat text file or a database right for an app?

One man practicing this “one-man band” concept is Paul Monies, database editor of The Oklahoman. He brings the data to stories across the paper, draws attention to data issues through his blog Data Watch and contributes to the open data movement in Oklahoma outside of the journalism world, and throughout all this, remains conscious of how data advocacy impacts his journalistic objectivity. Note: I interviewed Monies at the end of February, so the information is accurate as of that time.

This profile of Monies is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Entering the CAR world

Monies has spent time in his career as a copy editor and business reporter. He fell into CAR while in graduate school at the University of Missouri, and interned at The Oklahoman with database editor Griff Palmer, who is now at the New York Times.

He enjoyed the “puzzle” aspect of copy editing, his role at the College Station Eagle, which he entered after he completed undergraduate work at Austin State University, where he majored in communications. He was attracted by the need for attention to detail and the visual challenges of figuring out where different pieces of the paper would go on a page.

“I was still pretty young. I missed my friends, and having the weekends free, and I missed writing,” he said. “As a copy editor, I had that attention to detail and liked laying out the page, and thought it was a giant kind of puzzle, but I wanted to get back into writing and reporting.”

He finds his aptitude for visual journalism skills, as well as an affinity for attention to detail, to be beneficial in data journalism work. His visual aptitude serves him well as he does more and more with data visualization.

He left copy editing to go to grad school because he missed writing, and was looking for a more flexible schedule.

Advocating for open data

Monies attends various community meetings regarding the open data movement in Oklahoma, and writes about these issues in his blog Data Watch, hosted on The Oklahoman’s Web site. He said he’s aware that he must retain his objectivity as a journalist, but also this is an issue that he concerns him. At the same time, data-based journalists are some of the people most actively utilizing open records laws, and analyzing public information, so he also benefits from the progress that has been made. In the past, the Oklahoman and other papers would take action by publishing stories on how hard it was to get access to certain information that should be public. Monies felt himself drawn to go a step beyond that.

“I shied away at first from advocating anything to do with open records, or data, just because I thought, “Well, it’s maybe not my place as a journalist to do that.” But as I got along and started getting more into blogging, and seeing what else is out there, there’s really no one else that can sit up, in our positions who use that data on a regular basis, and do that kind of advocacy.”

He got permission from his bosses to “step out gingerly” and take a more active role on open data issues, especially as they related to public records.

The growing popularity of visualizations

As database work has shifted to include more presenting of information to news consumers, Monies said he has been eager to jump on board.

“I like seeing that side of data, too, and presenting data in different ways. A standard table gets pretty boring after a while,” he said.

Monies said he liked that this work brings him back to his design roots. He’s especially intrigued by tools that make it simple to create a visual explanation of data quickly — and he uses Many Eyes and Tableau Desktop.

He’s interested in moving more into data-driven applications, and hopes to continue to grow what the paper is doing in this area. He and some colleagues have been looking into Python and Django, and playing with a development server. But for now, Monies is using Caspio to post data online. As other reporters have said, it’s not because it’s his favorite option, but it works when there’s a small data team.

“It’s really easy to get into,” he said. “And if you just wanted to put a quick, searchable table up there, we’d use that. I can’t say that’s our long-term goal, but to me, it’s more of a stop gap and a way that we can reach to the next level, in terms of some of the more robust applications.”

Following in the footsteps of other data reporters

Monies’ post as database editor is not new to the Oklahoman. “I’m standing on the shoulder of some giants, and I’m really fortunate to do that,” said Monies.

He worked under Griff Palmer as an intern, who started the data program at the paper. After he left for the San Jose Mercury News, he was followed by John Perry, who is now at the Atlanta Journal-Constitution, and he was followed by Ryan McNeil, now at the Dallas Morning News. That means some records requests to public agencies have been standing for years . This both makes it easier to get information, but also brings up questions of whether this information is still needed, or the requests are taking up extra time, energy and physical space. Either way, Monies said the benefits of a data legacy are great, although there is still plenty of work to be done.

Data work is one of many demands on reporters’ time

Monies also works with fellow reporters on newsroom training, helping them to improve data and research skills to better integrate data into their work. Monies offers classes, and a local professor comes in to assist as well. But data is one of many skills reporters are being urged to learn, and the first priority for newsrooms is filling the paper and getting stories filed. So, often, the reporters just don’t have the time. Monies gets it, because he spent five years as a business reporter at The Oklahoman.

“I don’t let that hurt my feelings, because I was a business reporter for five years here, and so I understand the concerns with daily deadlines and weekly deadlines that everyone else has,” he said. “It was a struggle then to sit down and collect your thoughts on something new.”

Extended transcript

Read on for more of Monies’ experiences with data journalism, including the growth of Web development at The Oklahoman.

I’m curious to hear more about your experience with local data in Oklahoma.  How active have you been in pushing for it? Have you found a lot of kindred spirits?

I have found a few kindred spirits.  I started the blog about two years ago. We started a little data site that has some of the databases that we’ve used in our reporting for recent stories in the weekend edition, and in other places.  We started the blog as a part of that.  I shied away at first from advocating anything to do with open records, or data, just because I thought, “Well, it’s maybe not my place as a journalist to do that.”  But as I got along and started getting more into blogging, and seeing what else is out there, there’s really no one else that can sit up, in our positions who use that data on a regular basis, and do that kind of advocacy.  So, I got permission from my bosses here to step out gingerly and make a case sometimes, especially on open records.

I shy away from outwardly attacking politicians or agency officials, or anybody like that. I try and keep it polite.  But that was one of the things that I was concerned about when I first started with this advocacy side of it.  But it’s another outlet for us beyond running a typical story in the paper saying, “Woe is us, we can’t get access to these records,.”  For the public, I think it doesn’t really resonate a lot of times with them.  To me, as an industry, we don’t always make the best case for getting access to those records. So, I thought the blog would just be another avenue for presenting that information rather than a formal, dry story that we run every Sunshine Week for the last how many years.

As far as data in Oklahoma, my predecessors in this position have built a lot of groundwork with a lot of agencies.  So, we have standing open records requests.  I’ve got to credit people like John Perry and Griff Palmer. Griff Palmer, now at the New York Times, started the program here in Oklahoma, and then went off to San Jose, and now he’s in New York.  John Perry took over after Griff, as database editor here in Oklahoma, and worked here for a number of years, and then worked at the Center for Public Integrity, and of course now in Atlanta.  And then, Ryan McNeil was my immediate predecessor, who is now at the Dallas Morning News, and before that, he was at the Sun Sentinel.  I’m standing on the shoulder of some giants and I’m really fortunate to do that.

They’ve left behind standing requests for data that we still use to this day.  We still collect state financial data, and state payroll data, from a record that Griff Palmer first requested fifteen years ago, and we’ve just kept up.  We still have all that data.  Sometimes, it’s a question of, “Well, how far back should we keep it?”  Obviously, we maintain it monthly now, but that comes to be a question when you start talking about our tech folks saying, “Well, this server’s going to be done with, and we’re storing a lot of data on it.  What do you want to do with it?”  At that point, you have to make a decision . What are the benefits of keeping payroll data from 1995?  Data we don’t look at on a regular basis at all, but you don’t know if you might need it someday. That’s one of the questions that we’ve faced. I think we, here in Oklahoma, are moving slowly but surely toward opening up data in aggregate There are some stumbling blocks down at the legislature that happen almost every year.  Our session, we’re right in the middle of it right now, and there’s some buildup that would’ve opened up more data, but ran into some roadblocks on privacy issues and identity theft issues, which has been a kind of common complaint that lawmakers say they get from their constituents, although we try and make the case that we’re not going after Social Security numbers, or any super private information that we could use or misuse. We’re looking for aggregate information that agencies already have stuck in some database somewhere. Can they open that up and give it to people on a regular basis?

In the last year or two, we’ve gotten a pretty good local tech community here in Oklahoma City, and there’s a little pocket in Tulsa, too.  It’s people who are not journalists but interested in opening up data and studying open source languages like Python, Ruby and that kind of thing.  So, I’ve gone a couple times to their events.  It’s kind of funny being the only journalist in the room at some of those things and to see how other professions are approaching some of these data issues as well – it’s an interesting mix of folks.  Hopefully, that kind of community can grow, and maybe at some point, help advocate for data from local government.  We have a couple of lawmakers that are very well in tune.  The buzzword is “Government 2.0” right now.  They’re pushing for processes at the state government level, and we’ve got some good city folks here in Oklahoma City that want to do the same thing, but everyone’s running into the same issues right now with personnel and money and time.  Of course, that’s a common complaint in newsrooms across the country, and not just about government agencies.

Because you’re coming after some CAR giants, was a data culture already integrated into the newsroom when you got there?  How do you find you fit in the culture?

They were somewhat integrated, but like any newsroom, you’ve got reporters that specialize in different stuff, and some are not interested at all in data, some are fantastic writers but less concerned about the nuts and bolts of the statistical side of things.  One of my jobs, too, and my predecessors had the same role, is to help out in newsroom training, show folks basic spreadsheet skills, help them out with the latest things online, going beyond the standard Google search, really helping them find a mix of stuff on deadline, whether people or facts or whatever. We have a local professor who comes in once a week and helps out on the training things, too.  We have a decent amount of training available.  Now, of course, like any other place (I’ve worked at a couple of other newspapers, too) it’s hard for reporters to break off to get a concentrated period of time to do something that’s training.

It’s always a battle.  And it’s not necessarily from the reporter’s side that they want to do it, but their editor is over them, and wants to see a story every day, and the deadline needs to be hit.  They’re there to keep a seat filled in the newsroom while they’re gathering news, and that’s not always helpful, when you’re trying to sit down and explain the intricacies of  Access or Excel or statistical analysis.  There’s pretty good support for that, it’s just that people don’t have the time or interest sometimes to attend every event that you put on.  I don’t let that hurt my feelings, because I was a business reporter for five years here, and so I understand the concerns with daily deadlines and weekly deadlines that everyone else has. It was a struggle then to sit down and collect your thoughts on something new.

Can you take me through how your career path and how you got interested in computer-assisted reporting?

I’m 34 years old, I worked at my high school newspaper when I went to high school in Jacksonville, Texas in east Texas, but I was actually born in Scotland, but moved over to Texas with my mom and stepdad and sister in 1987. I took middle school and high school in Texas and then went off to college and worked for the college newspaper.  I bounced around between a couple of colleges early in my undergrad career.  I finally ended up as a student at Austin State University in Nacogdoches, Texas, and graduated from there in 1998, with a degree in communication and a minor in English. Then I worked for almost at the College Station Eagle in College Station, Texas where Texas A&M is.  I was actually a copy editor and page designer at that paper.  I enjoyed it, but didn’t like the hours as a copy editor.  I was still pretty young.  I missed my friends, and having the weekends free, and I missed writing, too.  As a copy editor, I had that attention to detail and liked laying out the page, and thought it was a giant kind of puzzle, but I wanted to get back into writing and reporting.  I applied to grad school,  and got into Missouri.

I attended Missouri from 1999 to 2001.  I took the standard classes on the news and print side. But it wasn’t until my first internship there in the fall of 1999, where I went on a whim and wasn’t really prepared, to figure out exactly what I wanted to do the next summer.  That’s one of my regrets, that I wasn’t as focused on internships at that point as I think college students are today.  I sat down with Griff Palmer at that time, who had been sent up by the Oklahoman to recruit interns for that next summer.  He mentioned at that time, while I’m at Missouri, they had an excellent computer-assisted reporting program at NICAR there, he said, “You should check out what they’re doing.” So, when it came time to register for classes in the spring, I registered for a computer-assisted reporting class.  Meanwhile, I’d sent off several applications for internships that following summer.  The Oklahoman called me back, and said, “Would you like to come be an intern?” and I said, “I would love to, but could I also maybe work with Griff on some data stuff?”  At that point, interns were rotating around the newsroom at the Oklahoman.

I spent three or four weeks on the copy desk, so I had some experience with that.  I spent three or four weeks doing data work with Griff Palmer, and got my very first computer-assisted reporting story done that summer.   I looked into the Coast Guard’s voting accident database, and did a little quick story on jet ski accidents in the Oklahoma lakes.  That came out in a July weekend edition that summer, and I was pretty excited about that, and pretty proud of that project, and Griff helped me out with a lot of that.  We also did an intern project that summer as a whole intern group, there’s probably about 15 or 20 of us total, and we did a project on open records.  We did an audit of every county in the state of four or five different types of open records.  It was us, and the Tulsa World that did the project that summer.  That was my entry into computer-assisted reporting, and once I went back to Missouri, I refocused and decided that was my career path.  I took more classes there, and then went off and did the Washington program for my last semester at Missouri.  My thesis was about how credit is assigned in computer-assisted reporting stories.

I did a survey of NICAR journalists about how they felt, if they were getting credit for their work that they were helping other reporters with.  It’s probably less of an issue now, but at that point it was a pretty big issue in terms of “Are you just a data jockey crunching numbers and then handing off the results to reporters who are getting the byline, and you’re only getting a credit line, where they couldn’t have done the story without your part?”  There were some varying degrees of concern with that at the time.  I haven’t seen it be much of a concern lately. I think everybody’s role has kind of morphed beyond just doing those parts.  We’ve got more database editors who are doing data analysis and writing, we’ve got more reporters who are doing writing and a little bit of data analysis.  Everybody’s fields have merged a little bit more, so I don’t know if that’s an issue anymore.  But that was definitely something I focused on in my thesis when I was a masters’ student.

What did you find in your research? Was it a big concern at that time?

You’re asking me about something I haven’t looked at for many years.  But briefly, of the 80 or 90 people who responded to the survey – and NICAR was a great benefit, and they gave me their mailing list, so I could mail out surveys, and this was before they had SurveyMonkey, so I’m dating myself, there weren’t online surveys you could fill out.  Well, maybe there were, but we weren’t using them.  I sent all these envelopes, and stuffed them, and put the stamps on them.  Basically, what I found out was that on the print side, among the people whose title at that time was database editor, there was a lot of concern about them not getting credited, and there were a couple of stories.

It was an anonymous survey, so I just followed up with people that gave me permission to follow up, but some anonymous comments were people missing out on prizes, which obviously is an integral part of a lot of journalism advancement and praise and career development.  People had missed out, they didn’t credit for a story they worked on, or a package of stories they worked on, there was some lingering bitterness over that.  But more than anything, there was just a realization that you have to fight for your byline, and fight for your credit line, because no one’s just going to hand it to you.  No one’s going to remember that you did that, unless you point it out to them.  You’re dealing with editors that are on deadline, and stuff is mislaid and forgotten about when it gets to crunch time on a project.  It’s just more of a cautionary thing, I think, from the survey, saying you just have to make sure that you’re credited, and you have to speak up for yourself.  It was an interesting thing to work on, and it gave me an idea of some of the pressures of that type of job.  Of course, I say that and did all this work in grad school, and came out in 2001.

I actually interned while I was in DC, and that last internship was with Dateline NBC.  I got a little view into the TV side of things, and thought about that as a possibility.  I basically worked in the Washington bureau of Dateline NBC when they were doing, I think, three to five shows a week, so they were really busy coordinating with the New York office, which was the main Dateline office.  But I was in charge of doing some of the data analysis for some of the stories.   I worked with a producer in New York, it was Andy Lehren, who’s now at the New York Times, at that time he was working with Dateline.  He helped me out a lot when I was working on the professional projects side of my Washington semester.  It wasn’t strictly related to my thesis but it was part of it as well.  I got a good idea of how TV news magazines handled computer-assisted reporting at that time.

I graduated from Missouri in 2001, and defended my thesis, and went off and became a cops reporter at the Waco Tribune-Herald, down in Waco, Texas, which I have no regrets about.  Coming from wanting to focus on computer-assisted reporting, there just wasn’t a whole lot of entry level computer-assisted reporting jobs at that time.  It was an interesting time in journalism.  It’s not as bad as it is now, but there weren’t a whole lot of jobs when I came out in that summer.  I went to Waco, and was a cops reporter for just over a year. I had a fantastic time doing that, I loved being out on crime scenes.  Tried to use some CAR skills while I was there, but mostly was doing regular cops reporting with a little bit of enterprise.  Looked at some jail inspections, that sort of thing.  It was a fun time to be a reporter, but it wasn’t exactly what I was focused on.

The Oklahoman called me up and said they had an opening on their business desk for a business reporter.  I told them, “It sounds interesting, but I have zero experience in business reporting.  I’ve never done it before, I’ve never really paid much attention to accounting, or anything like that, or even business while I was in college or grad school.”  But to me, I started thinking about it, and it was more of a challenge, and I thought, “Well, I’ll give it a shot, and see how I like it.”  And I really started enjoying it.  It was a lot to learn at the outset for someone who had covered mainly government and police before.  I covered some state capital and county government stuff while I was at Missouri.  But business reporting, to me, was this weird subset of reporting that I had no clue about and had to learn on the job.  Which I think, sometimes, is the best way to do it, to learn all these things, how to decipher a financial statement from a public company, strategies for getting people at companies, who don’t have to talk to you for any reason, to talk to you.   It was a little different tack than going after stories from government who are a lot of times compelled to make themselves look good, or tell you stuff under their open records act.  Business is a little different, because you’re dealing with the public relations side of that, and also a lot of small private companies didn’t have to tell you anything if they didn’t want to.  So that was more of a challenge for me.  You’ve got to say, “Well, can I do this?”  And I really started enjoying it.

I covered business for five years here at the Oklahoman.  Basically covered manufacturing, and some of the local public companies and the local economy.  So I covered unemployment and economic indicators and that kind of thing. It made it easier that I had a little bit of CAR background, so I was comfortable with numbers, and not afraid of dealing with them.  That made it easier for me to slide into the business beat, which is just so numbers driven when you’re doing a company earning story every quarter, or looking at financial statements every year.  It was something I didn’t shy away from, But then I was a little rusty in my CAR skills.  While I was there, we had two people, John Perry and Ryan McNeil, working as database editors, and so my skills got rusty, to be honest.  I was still pretty good with a spreadsheet, I could get into Access when I needed to, but when I took over as database editor, I had to give myself a quick refresher on some of the SQL skills that I hadn’t done on a regular basis for a while.  But it was like any other job, it’s just a process of managing what you can do right away, and a long-term plan.  I really need to get into this, and learn this, so that’s my goal for the next six months.

I’ve been database editor now for more than two years now, and sure, there’s still frustrations, like you deal with in every job with editors and deadlines and expectations, but I’d say I’m happy with my job 85 percent of the time.  It seems like there’s more to go after now in terms of data.  That’s been one of the things that I’m getting more into — the data visualization thing. Here, I’ve got to credit my predecessors too.  They have established that the database editor position has some pretty good tools to work with.  We’ve got Arc GIS, we’ve got SAS.  In the last year, I bought Tableau Desktop, which I think is a great little tool, just for some quick analysis. Nothing you couldn’t do already, probably, in Access or SQL Server, which we also use.  But if you want to visualize some stuff real quick, and point you in the right direction so you can run some more detailed queries later on, it’s great.

The data viz side, especially when it comes to Tableau and open source tools like Many Eyes, is great.  Tableau actually just released the free Tableau Public, which I’ve been excited to check out.  That’s all an area of passion for me right now, because I see these mashups that are on these other Web sites, doing all sorts of cool things with data and visualization, and to me, as someone who’s been a copy editor, and a business reporter, and a cops reporter, I like the visual side of things.  I liked it when I was a page designer, which I did briefly, it was less than a year, but I enjoyed it.  I like seeing that side of data, too, and presenting data in different ways.  A standard table gets pretty boring after a while.

Do you feel any pressure from management to expand into Web development? Is there an undue amount of pressure on your time?

There’s some pressure, but it’s mostly self-generated.  Our management is good. The way we do things here at the Oklahoman, we have a couple people in the newsroom that are pretty well skilled on the Web side of things.  The Web editors, our blog experts that can do WordPress and add-ons.  In the last year or two, we’ve looked more at a separate side of the news business that’s audience development, which focuses on the Web side. The newsroom is still putting out the paper, still contributing to the Web site, but we have another part of our company now that’s audience development and is focused on new Web platforms.  It’s not necessarily journalism-related, but we have this application and Web site that started out about a year and a half ago.  It started out as a directory and a calendar, and it’s now morphed into a local search engine for listings, and entertainment listings, and that kind of thing.  We’ve seen a lot of development on that from our IT folks.   And the newsroom side hasn’t really enjoyed that level of emphasis.  And I want to be diplomatic about it, because I don’t want to point fingers, I think we’re doing the right thing as a company as a whole, but I don’t think we’ve pushed very far into what the newsroom can generate in terms of Web development stuff. We’ve got some great people that do Flash animations, we’ve got some good things out of that.

But as far as data-driven applications, like you see maybe at other newspapers and other media companies, we haven’t gotten to that level yet.  But they have allowed us, meaning me and a couple other people in our newsroom, to spend some time on learning some other ways to do this stuff. We’ve got, including myself, three people who are doing some stuff on Django.  We’ve started doing that in the last year, we’ve gotten a little development server that we’re working on.  One of the Web guys in our newsroom has been working on some story walls, small projects he’s running through Django now.  My big goal is to learn a lot more about Django and Python specifically, and get some data-driven apps that we can replace Caspio with.

We’re using Caspio right now.  I know a lot of people, a lot of the purists, knock Caspio. Ryan McNeil bequeathed it to me, and now I don’t think he likes it very much.  Caspio, for all its detractors, when you’re a one- or two- or three-man operation, in a pinch, works really well.  It’s really easy to get into.  And if you just wanted to put a quick, searchable table up there, we’d use that.  I can’t say that’s our long-term goal, but to me, it’s more of a stop gap and a way that we can reach to the next level, in terms of some of the more robust applications.  But we’re hoping to move to Django.

Related posts you might enjoy:

• March 25, 2010 -- Self-teaching data and programming skills
• March 18, 2010 -- My next move: LA Times!
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• February 28, 2010 -- Django app #2: Conquering forms and Google Maps API
• February 20, 2010 -- Data Delver: Chase Davis, California Watch
• February 20, 2010 -- Data Delver: MaryJo Webster, Pioneer Press

As an aspiring programmer-journalist, Meyer personifies the very reason I love using data for journalism — it gives us the facts that hold up the truth. Society demands nothing less from the stories we impart, and I know I think of Meyer’s writings nearly every day. Back in February, the now-retired University of North Carolina-Chapel Hill professor took the time to speak with me. Hopefully, he’ll inspire you like he’s inspired me.


This profile of Meyer is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Pioneering computer-assisted reporting

There was a time when the term computer-assisted reporting meant something special. The very notion of “regular people” using a computer was brand new. Journalists embraced this technology before they could just Google how to make it work. One of the first investigative reporters to use computers to dig into data was Phil Meyer. He used database techniques to enhance his analysis and practice journalism that lived up to the standards applied to the social sciences.

Journalism education should include math

Meyer created a test that he thought all students should have to take to earn a journalism degree at Chapel Hill, where he taught for many years. All students took a spelling test. There was a certain minimum requirement to be admitted to the school, and a higher minimum requirement to graduate. The test required about a sixth or seventh grade level proficiency in math.

For Meyer’s part, he taught several courses at Chapel Hill, integrating math into many of them. Many members of the CAR community remember Meyer’s precision journalism class, which he taught for decades. ” I think in the 24 years that I taught, the interest declined,” he said. “If you want to study without learning math, at this school you go to the art department, not the school of journalism.

Precision journalism

Meyer said he believed so strongly in the technique of bringing quantitative analysis techniques to journalism that he had to put his thoughts down on paper. “It was just something I had to get off my chest,” he said. This resulted in a book called “Precision Journalism” that showed how statistics, the scientific method and databases could be applied to journalism.

Meyer cited Steve Doig’s work at the Miami Herald as some of the earliest pieces that made the most of providing valuable information for using computers and databases for analysis. “Steve Doig did some programming that let users go online and compare their property values with those of their neighbors and look at the different characteristics that led to those evaluations, and then they could make up their minds whether they wanted to challenge theirs. That’s pretty useful,” Meyer said.

He also praised the USA Today series “The Smokestack Effect,” awarded the first place 2009 Philip Meyer Journalism Award at the most recent NICAR conference in Phoenix in March 2010. “It brought two databases together, and drew conclusions that weren’t obvious to the naked eye.

Pro-journos serve a necessary function

Programming-journalism was how Meyer first discovered precision journalism, just as it’s how many journalists discover the power of data-driven presenation. Meyer said the difference is he used coding more for analysis, while modern pro-journos focus more on presentation, but both sides are highly essential. “It doesn’t do much good to develop complicated information if you can’t get it into the consumer’s head.”

Don’t do programming just to do it

When new technology combines with journalism, sometimes there’s a tendency to use it just to show off, Meyer said. He said that this might happen again, just as it did when computers first became accessible to journalists. “When there is a new technology like that,” he said, “people will use it just to use it, and maybe forget the journalistic purpose of it.”

Extended transcript

For more of Meyer’s thoughts on precision journalism as practiced in the modern era, read on.

When did you stop teaching at Chapel Hill, and what were you teaching in your last classes?

I retired July 1, 2008, and my last course was advanced reporting, which when I teach it, is basically precision journalism.

What did you see in terms of people’s motivation to abide by the standards put forth in your book, Precision Journalism? Has it gotten lower, higher, stayed the same?

The idea was pretty radical when I wrote the book, and there was pretty rapid acceptance, and then there was a leveling off. I think in the 24 years that I taught, the interest declined. If you want to study without learning math, at this school you go to the art department, not the school of journalism. I never did succeed in persuading my faculty colleagues to have a standardized math test be one of the conditions for graduating. We did have a standardized spelling test that everybody had to pass. In fact, they had to pass it twice. They had to get one score to get into the school, and a higher score to graduate. I would have liked to have done the same with math.

How advanced would such a math requirement have been?

About sixth or seventh grade math. In fact, I have online the sample of the math test I would have given.

When you wrote the book (in 1970), where did you expect the precision journalism movement to be by 2010?

I had no idea, it was just something I had to get off my chest. And I was encouraged to write it by the Russell Sage Foundation, which had a slightly different agenda. It was mostly run by sociologists, and they were interested in the application of quantitative methods to sociology, which was just then getting a big boost from computers. And they thought if journalists used these methods, then they would appreciate quantitative sociology better.

Do you think the prevalence of computer programming in journalism nowadays helps with precision journalism?

The higher-level languages are essential. In fact, that was how I got started was with one of the very early higher-level languages. It was called Harvard Data-Text, and it was a forerunner to SPSS, and without that, I’d have been sunk. SPSS, of course, is written in Fortran, and Fortran translates to the assembly language, and that translates into the basic machine language. Harvard Data-Text would have been as popular as SPSS is today, except it was written in the IBM 7090 machine language, in order to conserve computer time, which was very precious then. That meant when the 7090 became obsolete, they had to start over again and write it for the next generation of computers. They lost a lot of time, and SPSS pulled out ahead of them and became the standard for academic, statistical computing for a long time.

As one of the first people to use a computer to do journalism, what do you think of the modern trend of the rise of the programmer-journalist?

It is new in that the programmer-journalist of today is less interested in data analysis than in presentation, and that’s an important area, too. It doesn’t do much good to develop complicated information if you can’t get it into the consumer’s head.

Do you think they should also be looking at data analysis?

Well, yeah, but this is an age of specialization. I’m not sure everybody should be able to do everything. We need good managers now to coordinate all of those specialties.

What’s your overall impression of the computer-assisted reporting you are seeing coming out of the media now?

Oh, it just keeps getting better.

What are examples of some of your favorite recent projects?

It isn’t that recent, but one of the best of all time was Steve Doig’s examination of the relationship between destruction and Hurricane Andrew and enforcement of building codes. The hurricane building code was enforced just before I first arrived in Miami as a reporter in 1958. I remember the house my wife and I bought was one of the first built under the code. Then, as time went by, corruption and carelessness made the code weaker and weaker. Doig noticed that newly-built houses were much more badly damaged than old houses. I checked that out myself. I was headed to South American for a day, and I stopped in Miami and looked around. I went by my old house, and it was in incredibly good shape. There was a fairly new development a few blocks away, and it was devastated. So, Steve was able to document that with maps and statistics. He was a SAS programmer, and it was a very compelling story, and it won the Pulitzer Prize. Now, there was a time when the Pulitzer board would not award a prize to a story that was based on computers, because they figured that was cheating.

When did that stop?

I think the first to win the prize was Bill Dedman’s “The Color of Money.”

Why did they decide to allow it?

I think it just became more accepted as a standard reporting practice. I guess if everyone used an aluminum pole in pole-vaulting, then it wouldn’t be a problem.

What’s your reaction to the movement of uncontextualized databases?

I have mixed feelings about that. I’ve seen some stuff that’s just using computers to show off. One example that I hated the most was a newspaper examined public records on traffic arrests and looked up the blood alcohol content so they could identify the state’s drunkest driver and go interview him. I can see using data like that to look at trends and patterns, but to pick out someone for ridicule doesn’t seem like a good idea. When there is a new technology like that, people will use it just to use it, and maybe forget the journalistic purpose of it.

Was that something you saw happening often?

I haven’t seen very many recent examples. But as computer techniques become more and more powerful, we might be in for another wave of that. Now, it’s possible to look at public records and dump them on to the Internet in a way that’s useful. Steve Doig did some programming that let users go online and compare their property values with those of their neighbors and look at the different characteristics that led to those evaluations, and then they could make up their minds whether they wanted to challenge theirs. That’s pretty useful.

What are your reactions to the three stories that won the Phil Meyer award this year? Are you pleased with their quality?

Oh, yes. One was the story that won the Grantham Prize award, I was on the jury for that, although I had to abstain because of my previous relationship with USA Today. That was a really nice piece of work because it brought two databases together, and drew conclusions that weren’t obvious to the naked eye. The schools were placed where the pollution was, it wasn’t pollution going where the schools were, evidently because land in those areas was cheaper.

Related posts you might enjoy:

• March 28, 2010 -- Data Delver: Mark Schaver, Louisville Courier
• July 11, 2010 -- Data Delver: Andy Boyle, St. Petersburg Times
• April 11, 2010 -- Data Delver: Paul Monies, The Oklahoman
• April 5, 2010 -- Women with coding skills are no Passover miracle
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica

Entering CAR for investigative work

Mark Schaver first discovered his love of investigative reporting as a graduate student at the University of North Carolina – Chapel Hill. He took a business reporting course with Phil Meyer. Soon after, Schaver worked as a reporter for the Louisville Courier, in the Paducah bureau, and later moving to Louisville. Throughout his reporting, he observed that databases were a way to enhance investigative reporting, and advance his career.

Technology wasn’t the primary reason he got into CAR — for him, it was all about the investigations. “I originally got into CAR because I was interested in investigative reporting, and that was an obvious way to make yourself stand out,” he said. “And I did have an interest in computers.”

Shifting nature of CAR

But in recent years, Schaver, who used to direct CAR at the Courier, said his role has changed significantly. “I’m filling in for the night editor who’s been on vacation and on furlough the last couple weeks, so I’ve been working every night, so I’m not doing much CAR stuff at all,” Schaver said.

It’s still a far cry from the investigative work he loves to do, because he’s being pulled in many different directions. He attributes some of that to a shifting of the CAR specialization. “It had totally evolved from a reporting, journalistic emphasis to a Web development, data munging for the Web kind of thing, ” said Schaver. “Parts of that I like, I’ve learned a lot about Web programming, but I don’t really want to be a Web developer.”

Data on the Web requires organizational support

He supports data-driven applications, but said your organization really needs to be behind it to do it well: “If your organization is committed to that sort of thing, I think it could be more interesting, but not necessarily when you’re a lone wolf trying to do a hundred different things, and you have no time to really develop things.”

He suggested that it might be easier if large newspaper chains arranged a central data team that could work on these projects for multiple papers, but that it becomes difficult when one reporter is the data expert trying to do CAR, frameworks, investigations, etc.

Using Caspio, a quick-hit data posting solution

Time is limited, and he said that’s part of what drives him to quick solutions to posting data. Caspio is one of the most common quick hit solutiiosn used in the journalism world. Schaver said he understands where others are coming from who believe we can do better (see Derek Willis’ posts here for some quick bullet points, or here for a longer explanation, to learn more about this viewpoint).

But for Schaver, he does what he has to do. “I don’t want to do it, I can do something better, but it’s like, I could turn it into an application if I had a couple days, but I don’t have a couple days. I got an hour, and I’m going to throw it into Caspio, and get it off my plate, because it’s just not worth more time than that, so Caspio fills that need.”

It’s not all about data

Schaver thinks data is an important component to journalism, but it isn’t all encompassing. “I’ve been thinking that most stories don’t have something that data will bring anything to in any reasonable time frame, data doesn’t even enter into the journalism,” he said.

But if you want to go into that world, Schaver recommends it as a good option. Yet, he’s careful to offer the caveat that it’s just one option, or journalistic subset, and it’s not the entirety of the future of the industry.

“There will also be people who are able to tell stories through video and audio and with multimedia and with databases, and that will be a piece of it. But the database part of it is just a very small aspect. It’s a significant one, and it’s new, and it’s interesting, and there will be a lot of cool things done, and if that’s your passion, you should do it, but I just don’t think it’s the main thing that’s really going on out there,” he said.

Extended transcript

An adapted transcript of Schaver’s reflections on his work, and the role of data journalism, continues below.


Your role at the paper has changed recently. What is your day-to-day work now?

For much of the last decade, computer-assisted reporting director was my title, and I dealt with computer-assisted reporting. Sixteen months ago, because of all the downsizing, they ran out of editors and said, “Well, we wonder if you could fill in and be the Sunday night Metro editor temporarily.” And then it became permanent. And then last summer, they asked if I would be Assistant Metro Editor, about half the time. So essentially I’m the night Metro editor a couple of nights a week, and then late in the afternoon I help move and edit stories when the big rush is on, and then the rest of the time, I’m supposed to do computer-assisted reporting stuff. That’s the situation now, although in practice, the way it works is that the editing thing just takes over more and more of your time, so I’m probably spending most of my time on editing duties. Like this week, I’m filling in for the night editor who’s been on vacation and on furlough the last couple weeks, so I’ve been working every night, so I’m not doing much CAR stuff at all.


How do you feel about that?

I was actually glad about it because I wanted to get back into journalism more, although I have mixed feelings about it. I originally got into CAR because I was interested in investigative reporting, and that was an obvious way to make yourself stand out. And I did have an interest in computers. That was my original intention. And in the time I did this, initially we had an investigative reporting desk, and I had an office next to them, and I worked with them a lot. And I was learning stuff, and it was all new. And then, over time, as the media has shrunk, the investigative reporting team was disbanded, and the editor had left, and the job evolved more into putting databases on the Web, and doing Web-related stuff. It had totally evolved from a reporting, journalistic emphasis to a Web development, data munging for the Web kind of thing. Parts of that I like, I’ve learned a lot about Web programming, but I don’t really want to be a Web developer. So I was wanting to find a way out of the job that I was doing, and I was feeling a little trapped, because once you develop these skills, you’re hard to replace, so you have to find someone else who will do this stuff, which is even harder when we’re downsizing.


What type of skills do you see as being in such high demand?

We’re a mid-size newspaper, we don’t have a Web programmer. We have a guy who does PHP work, but he has a lot of other online duties, and that’s it. We don’t have a dedicated Web developer, like some papers our size do, like Indianapolis, who’s also run by Gannett. Some papers have chosen to hire people whose full-time job is to do Web programming. We don’t have, per se, a Web designer, so you’re doing that. In my time as a CAR guy, I created the internal budgeting application that we use to budget stories and assign photos and graphics. That’s not really a reporting function, that’s usually the thing that might be done by an IT person. That had started for one reason, and then grew and took over all these other functions. Recently, we adopted a new publishing system that actually has that functionality built in, so it’s gone away. You’re the one who built it, who programmed it, who knows how the databases work. If you get rid of you, they’ve got to have someone else who is going to do that stuff. I do a lot of the mapping with ArcView, I make maps for the Web, for stories, and I hand them off to graphics. That’s a skill, somebody has to do it, if we’re going to do it. The skills cover a gamut of different things.


How did you first discover CAR?

I went to graduate school at Chapel Hill in the ‘80s, and one of the professors there was Phil Meyer – that must have been where I first heard about it. I read his book while in graduate school. I took a course with him, although actually I couldn’t’ fit his reporting course into my schedule, I took his business journalism course. In the early ‘90s, I got my first computer, when I was a reporter here. I was a Western Kentucky reporter, so I lived in Paducah. That was pre-Internet. Back then, people were using spreadsheets and databases, and I’ve paid attention to it ever since. It was obvious all of the investigative-type projects would have some kind of computer-assisted analysis or component to it. So it was a niche that, because I was interested in long-form reporting, and because I was interested in bettering myself, I wanted to pick that stuff up over time. And so, I did.


Are your skills mostly self-taught?

I went to a few NICAR conferences, but I don’t go every year, often for money reasons, and I’m not a lover of conferences, as much as some people are. It’s a useful way to get to know people, and they’re always inspiring for me to see what people are doing. I haven’t had any formal training. The only IRE-type course I had was when I first became a CAR guy, I took a mapping course at Missouri.


Where did you go to undergrad?

Boston University. I was a political science and economics major, and I had no intention of doing journalism at the time. I was interested in becoming a reporter, I was interested in politics and current events. After I graduated, I worked various odd jobs in New York and elsewhere, and traveled overseas and in the U.S. I decided at some point I wanted to be a journalist because it would allow me to keep traveling for free, and because I wanted to write.


What did you do after graduate school?

I came to the Courier in 1990. At the time, we had bureaus, this is speaking of diminishing horizons of newspapers. In the ‘90s, we had bureaus all across the state. I was at a bureau in Western Kentucky, and I covered 19 counties. We even had an airplane that would fly people to the scene of important stories far out in the state. I was a Paducah reporter for three years, I actually followed my wife out there, by the way. I moved to Frankfurt, where I became an education reporter, and also covered the General Assembly. Then, I married my wife, who was also a reporter in Frankfurt. We wanted to have kids, so we moved to Louisville, so she could essentially go part-time. I became a police reporter and courts reporter in Louisville, and then I was asked if I wanted to do this job, the CAR job, which I actually don’t have anymore. I was Computer-Assisted Reporting Director for nine years, a title I never liked, because it was really hard to explain to people what you did. I reported to the managing editor for most of that time. Gannett got into, in a big way, renaming the newsroom, we’re not a newsroom, it’s the information center, and reconfiguring things and creating what they called data desks, which is essentially what used to be the clerks in the library, so they attached me to that for a while. When they asked me to do editing, they moved me back to the Metro desk. Now, Assistant Metro Editor is my title.


What do you think of the concept of the uncontextualized posting of data online?

It has its uses, but it’s not very interesting. I love Matt Waite’s term, data ghetto, which is true. To do it well takes a lot of time and effort and skill. It’s something lacking, that newspapers can’t do easily, The stuff that’s interesting is the stuff that people will spend a lot of time with. Then you have to justify the amount of time we spent on this project, is it better to do that, or is it better to produce copy and stories every day? I’m not the biggest enthusiast. I’ve done it, and I’ve tried to get better, but I don’t think I’m the best person at it. I think it’s better to have someone who’s a skilled programmer, not a self-taught one. Somebody who has a real passion for it ought to do it. I tried to do the best I could, and I’m still doing it, but it isn’t my passion, per se.

I’m more into the information gathering, and the investigative side, and analysis stuff that’s intellectually interesting to me. I enjoy the other stuff, but I don’t want to do it full time, and searchable lists are what they are. We’ve always published agate in the newspaper, and there were always people who would passionately go through the box score, and look at that stuff, and that’s great. But that doesn’t interest me that much. If we were doing a big project, something like Politifact, that was a project that involved lots of reporters, designers, editors, had top-level buy-in from management, and they worked on it and committed to it, and if your organization is committed to that sort of thing, I think it could be more interesting, but not necessarily when you’re a lone wolf trying to do a hundred different things, and you have no time to really develop things.

The New York Times, they do beautiful stuff, but they throw five, six really talented people at it, and spend a lot of time on it. That’s great, but I don’t think that’s a scenario that will work for most news organizations. I don’t know why the chains that have 90 newspapers, or dozens of newspapers, they haven’t done more to create news-type applications across organizations instead of letting each organization do it individually – it just doesn’t make much sense to me. I went to a NICAR conference in Cleveland a few years ago, and every session I went in, it seemed like it was a New York Times reporter. I went through the program, and added them all up, and it was like, 13 percent of all the presenters were from the New York Times. I thought this is a real distortion.

It was also the meeting where they had the Caspio guy, Caspio had really started pushing in the newspaper world, and Derek [Willis] is famous for not liking Caspio. He’s right in the sense that it’s better if we do it individually. It would be great if your organization commits to it 100 percent, but most people are working in a scenario where you’re balancing all these needs, and throwing it into Caspio is easier, I’ve done it recently. I don’t want to do it, I can do something better, but it’s like, I could turn it into an application if I had a couple days, but I don’t have a couple days. I got an hour, and I’m going to throw it into Caspio, and get it off my plate, because it’s just not worth more time than that, so Caspio fills that need. People are still going to do wonderful things, even in small places, and there’s still room for that, but I don’t think it’s the main river of where we’re going these days.


Where do you see the future of CAR?

If you’re in this CAR/data world, it is a new aspect to journalism. I don’t really agree with the notion that the world needs more programmer-journalists, I haven’t been convinced of that. It’s nice to have people with technical skills and an understanding of that, and it would better to have more people in journalism understand that, but the data side only applies to some stories. It’s great for me to be editing stories again, after ten years of basically thinking in code, and writing a few blog entries. To sit down and edit stories again, I’ve been thinking that most stories don’t have something that data will bring anything to in any reasonable time frame, data doesn’t even enter into the journalism. Most stories, it doesn’t apply to what we do. There is a class of stories where it does apply, and it’s great to bring it to that, but there are many, many other stories it doesn’t.

And to me, if you look at what’s succeeding on the Web, I really don’t think it’s the Everyblocks of the world. It’s a cool thing, and it’s well done, but what’s succeeding on the Web are people who are really good bloggers who can produce a lot of content, cover a lot of ground and update frequently, write with a voice with personality, who can interact with people on the Web, and in social media, and who can extend their brand and their name and their reach to an audience, that’s to me what’s the most important thing. That’s what succeeding.

The architecture underneath the site isn’t that important, but there will also be people who are able to tell stories through video and audio and with multimedia and with databases, and that will be a piece of it. But the database part of it is just a very small aspect. It’s a significant one, and it’s new, and it’s interesting, and there will be a lot of cool things done, and if that’s your passion, you should do it, but I just don’t think it’s the main thing that’s really going on out there. It is, if you live in this data world, you tend to see everything as data, and how important it is that we should really be embracing this. You can, but I just think it would be better to have ten people on your staff who can write with a voice, who can update their blog frequently, and keep reporting, chase a good story for weeks on end, updating continuously, and interacting with the readers who call and write in, or comment on Facebook and Twitter. I’d rather have ten of those people, than ten data geeks. So that’s just what I mean. I just don’t think it’s the future of journalism.

Related posts you might enjoy:

• February 20, 2010 -- Data Delver: Chase Davis, California Watch
• February 13, 2010 -- Data Delver: Mo Tamman, Wall Street Journal
• April 5, 2010 -- Data Delver: Phil Meyer
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• March 8, 2010 -- Data Delvers: Ben Welsh & Ken Schwencke, LA Times

This profile of Wynn is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Power of a small paper

Wynn said he values his experience working with data at the Springfield News-Leader.  He got the job, serving as an editor, dirctly after graduating undergrad at the University of Missouri, where he had worked with NICAR.  At Missouri, he worked with fellow NICARians Chase Davis, now of California Watch, and Ben Welsh, now of the Los Angeles Times.  Mike Pell, now at the Center for Public Integrity, was also in that group.  He said smaller papers are essential for learning all the nuances you can’t be taught in school, and learning lessons on a smaller scale.

“It was really fun, and I’m so blessed I got the chance to go there and work with as much data as possible.  I could push the boundaries without having to worry too much about failing.”

Bringing interactive data to Phoenix

After a year in Springfield, Wynn moved to the Arizona Republic.  He described his job as “taking public info and making it as easy to use as possible.”  When he posts data, he tries to make it accessible to the “lowest common denominator.”

“You want people to get as much or as little information as they want or need,” he said.

Posting data allows the story to live- -and Wynn sees this as an extension of more traditional CAR.  “More and more CAR is becoming more about development.  The traditional CAR output is a story, and the goal is to get people talking about something.  When you post data online, you create an application, the information lives and breathes and is more long-term, it feels like it has more of an impact.

According to Wynn, interactive data pieces are popular, and produce pretty constant traffic.  He thinks one of the main factors that helps is that data applications are continuously and automatically updated, so many data features constantly have fresh content.

Bringing data to reporters

Finding story ideas and instilling data culture in the newsroom is another key component of Wynn’s job, he said.

“One success is that we’ve really made people a lot more receptive to data, they see the possibilities. We can pitch ideas, but now, sometimes, reporters come to us with ideas,” he said.

Coding needed for journalists?

To get the tools to post data online, Wynn has dabbled in both Django and PHP.

Wynn said that he advocates journalists learn at least some coding, and if you don’t know something about it, it remains a mystery.  “If you’ve never dealt with it, it can be scary.  But it’s a tool, and you need to understand what it can do.”

Related posts you might enjoy:

• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• March 8, 2010 -- Data Delvers: Ben Welsh & Ken Schwencke, LA Times
• February 13, 2010 -- Data Delver: David Donald, Center for Public Integrity
• February 20, 2010 -- Data Delver: MaryJo Webster, Pioneer Press
• February 20, 2010 -- Data Delver: Chase Davis, California Watch
• February 15, 2010 -- Data Delver: Lisa Pickoff-White, California Watch

Path to CAR

DeBarros first became interested in journalism while in college, and was considering going into the radio business. He worked at a local station while in school, but ultimately decided against it as a career due to the low pay.  So, he started work as a cops reporter.  At about the same time, he bought his first personal computer and enjoyed experimenting with all the different tools it had to offer.  “I reformatted floppy disks for fun,” said DeBarros.

So, when the time came for newsrooms to start more thoroughly integrating technology into their daily routines, DeBarros said he was ready and willing to get his hands dirty with any gadget that he could. He served as systems editor, a common position in the ’90s, helping the Poughkeepsie Journal to transition to color printing, and working to retool their page layouts.  He also worked with the Life section of the paper, in his editorial function.

Technology and journalism — it was a combination that DeBarros continued to notice and enjoy.  So much so that he went back to school for a computer science degree, thinking it would be interesting and help his career, he said. While earning that degree, he learned about databases.  And he continued working at the Poughkeepsie Journal.  Soon after, he went to USA Today on a loanership program — Gannett properties such as the Journal would send staffers to Washington to gain experience on a national paper.  DeBarros met the other database editors, including Census guru Paul Overberg, and DeBarros never left.

USA Today’s multi-section approach to CAR

When DeBarros first started at USA Today, the four database editors who made up the CAR team all worked together. But by 1998, they spread out into various sections.  For example, Barbie Hansen moved to Money, where she remains today, and DeBarros focused on Life, especially entertainment.  He remained there until 2008, when he became Senior Database Editor, and works with other CAR specialists across beats.  DeBarros said USA Today has made a commitment to spreading CAR across topical areas, since there are data sets that are applicable in a large variety of stories.  “We’ve had the relative luxury of having more than one CAR person or database editor,” said DeBarros, “so we made the decision early on not to focus all of our people on those traditional news section topics, but spread out into the other areas as well.”

Data’s essential to the story

Interview a data source the way you would a human source, said DeBarros.  Not all data is good data, but it’s up to you as a journalist to figure it out. “There are some data sets out there that are rich and deep and valuable, and you’re going to go to them again and again,” he said, “and there are some where you’re like, ‘Meh, doesn’t really do much for me.’ But the only way to discover that is to really interview them and find out what they’ve got, and what they’re really telling you.

He’s concerned about news organizations providing uncontextualized data sets, but if you approach the use of data philosophically, following in the footsteps of many NICARians, you can’t really separate the data from the context.  “We’ve never really divorced the notion of context from data, because to us, data was never just a means to an end in itself, it was always the driver of a story.”

Changing platforms

DeBarros sees the rising popularity of all sorts of technological tools for analysis and presentation.  He said a lot of that is currently falling to the CAR specialist, and it’s an extremely broad spectrum for one person to cover.

“I think it’s very hard for people to be an expert in mapping and an expert in frameworks and an expert in statistics and an expert in data parsing and an expert in writing – there’s a lot there,” he said.  “Right now, I think we have a lot of people who are doing their very best to try to cover all those bases, but people being people, we all gravitate toward different parts of that spectrum.”

But the most important part is to remember that no matter what the tool — it’s still always about conveying information and telling a story: “I just think we always need to think about: What is the story that we’re telling? There always has to be a story. And that holds true no matter what we’re doing.”

Extended transcript

Keep reading for more about the dovetailing of DeBarros’ career and the development of computers as well as the type of work CAR reporters do at USA Today.

Trace your career path from how you got started to CAR and journalism to how you got to USA Today.

I was always focused on the arts and liberal arts. Even when I was in high school, I gravitated toward the school newspaper, and then the radio station. In fact, I was very much set on a career in radio and broadcasting. After my first two years of college, I wound up getting a job at a local radio station, working as a disc jockey at a big rock station in the early 1980s. I did that for a little while, and I knew that I was going to go and finish off my degree, and I was trying to decide, because I had studied communications for the first two years of college, I wasn’t too sure if I wanted to continue as a communications major, or if I wanted to declare a major in something more specific, because after working for radio for a little while, I realized it was an extremely low-paying job. In fact, I was making more working at a local McDonalds than I was working in radio.

I had always gravitated toward writing, and I’d always really loved telling stories, I put my own little newspaper together as a kid. So, I decided to go back to school, and declare myself an English major, and do a concentration in journalism. So, I was going to Marist College in upstate New York at the time, and what I also did, while I was going to school and doing that degree, was the radio station let me start doing some work for them in their news operation. It was a small radio station, and they had two people who were their basic full-time news staff, but they needed somebody to go out and cover stories at night and go to town board meetings. So, that’s how I really got into journalism.

I graduated from college and got my first job as a police reporter and obit writer at the Poughkeepsie Journal in upstate New York, which is a Gannett paper, it’s still there. That was in 1986. It wasn’t so long after that, that I went out and bought my first personal computer. That was in 1987 or 1988, it was an Epson. It came with no hard drive, but it did come with two five-and-a-quarter-inch floppy disk drives. So any program that you wanted to load, you had to insert the disc for that particular program in the drive, and let it load that way. This fascinated me, because I had taken some computer programming courses in college, and I had a bit of an aptitude for it. I really liked typing in codes, and seeing what the computer would do. As soon as I had that PC, I really started to devour the manual that came with it. The operating system was DOS. I started going through the manual, and going through all the different commands. I reformatted floppy disks for fun.

Then, I went out and bought a hard drive, and put that into the system. I started learning how to use the little spreadsheet program that came with it to put numbers in and add them together. And I just was like, “Wow, this is really cool. You can do a lot with computers.” As my career was developing as a reporter, and as an editor, in the newsroom, I naturally started to gravitate toward any kind of technology that the newsroom was implementing. I found myself on committees to put in new desktop publishing systems.

There was a whole job function that developed in journalism in the 1990s. A lot of newspapers had something called a systems editor. They don’t really have them anymore. But the systems editor was the person who helped the newsroom figure out what to do with emerging technologies. You have to understand that newspapers were going through a transition from black and white to color printing. USA Today had come along with color, but a lot of newspapers didn’t follow suit for many, many, many years. They needed printing press upgrades. The Poughkeepsie Journal didn’t go to color printing until some time in the mid-1990s. At the time, newspapers were also moving from the mainframe, dumb terminal model of systems to the desktop publishing system. I wound up being the guy who set up our Macintosh network, and programmed all of the templates for our page design and all that. I was really, really interested in anything technological that the newspaper would let me get my hands on.

In the meantime, I was the life editor for a while, I was on the copy desk for a number of years, I was doing all different kinds of jobs.As I started to get more interested in technology, I decided, “You know, this actually could be a career path for me, and I’d like to get some formal training in it, so I went back to college part-time. I said, “You know what, I could get a masters degree in computer science, and that would probably be a good career move.” I went back to Marist College on a part-time basis. It took me about seven years altogether, but I got a masters degree in computer science. In the course of that, I learned a lot about programming, a lot about databases and systems design, and all of that. It was during that time that I started to get really involved in doing demographics research for our newspaper.

There were some coverage things we were doing, we wanted to expand into different markets, and the publisher and the editor came to me and said, “Hey, could you do a market study for us, and could you figure out where the population is growing, and where it’s not? What might be some opportunities?” So, I started to get familiar with all the data that was out there in terms of the Census and economic data. I started to do some profiling of our area, and that very naturally led to me taking over one of the only personal computers that were in the newsroom at that point, and turning it into a real basic computer-assisted reporting work station. I stuck a copy of Paradox on there, which is the database program that a lot of us used before we moved into Access and SQL Server. Paradox was very cludgy, it wasn’t the type of thing you wanted to let your life revolve around for very long. A lot of things came together between my natural training as a journalist, and my interest in technology, and a lot of different opportunities that kept coming my way.

It wasn’t too long after doing all of that, that a reporter in the newsroom came to me and said, “Hey, it would be really good if we could figure out what the most valuable properties are in the city of Poughkeepsie. And I thought to myself, “You know, this might be a good opportunity for me to go and make friends with the IT guy over in City Hall.” I went over and visited him, he was down in the basement of City Hall, in the computer room. Back in those days, they all had big mainframe computers in an air-conditioned room. Actually, what I first did was I went to the tax assessor’s office, and I said, “I want a list of all the properties in the city of Poughkeepsie, and how much they’ve been assessed for.” And they pointed me over to the corner, where there were these big books, filled with computer printout, and they said, “Well, all the numbers are there, and you can just start copying them down.” And I thought to myself, “If they were printed on this piece of paper that looks like computer paper, then certainly they are in a computer somewhere in this building. And I can get that data on a disk that I can bring over and put into my computer.” And that’s how I really started figuring out that we can do computer-assitsted reporting by going to the government and getting data.

That’s what I did. I went to visit that guy in City Hall, and I said, “Look, I know you’ve got a file on your computer. I’d love to have you put it on this floppy disk for me.” And he had to check with the local attorneys, and get their permission, and I called up a sunshine advocate in New York state and got him to weigh in, and they agreed that, “Yeah, the law says we can do this.” The next thing I know, I had that data on the computer, and was going through it in Paradox. We wound up writing a couple of stories about different properties.

I had the opportunity to come down to USA Today in 1997, as part of a loanership program that Gannett used to have, they would bring reporters and editors from the various Gannett newspapers into Washington D.C., into USA Today for four months. They’d get the experience of working for a larger newspaper and then go back home. So I came, and never left. I got involved with what was then called the enterprise department. There were three other database editors, one was Paul Overberg, he’s our Census guru, and there were two other people. They hired me, and that’s where I’ve been ever since. The loanership program started pretty early on, around the inception of USA Today in 1982. They ended it years ago, probably around 1999, 2000 or 2001.

After you arrived at USA Today, how did you get to be the head database editor? How has the structure of CAR changed?

When I started, we were all in one section, and we were called enterprise, and we were basically a special projects team. There was myself, Paul Overberg, Barbie Hansen, who’s still at USA Today, and another woman. The four of us basically worked on the projects as they arose, but within about a year, the editor of USA Today at the time decided to end that department, and sent each of us to one of the four sections of USA Today. Because I had a long history with covering entertainment and features and education, I volunteered to get attached to the Life section. In about 1998, all of us left that projects department and went into the various sections. Paul went to News, Barbie Hansen went to Money where she remains to this day, and the other person went to Sports. They left that position eventually.

I spent from 1998 until 2008, a good ten years, focused on developing computer-assisted reporting around the topics of health, education, religion and all things entertainment. By that I mean, movies, music, books, television, Broadway, and really,nobody at that time, and nobody since, has really done a lot of CAR around entertainment. I think there’s been a few things, but nobody on a consistent basis. I think USA Today’s been somewhat unique in that. Just because we’ve had four database editors, now we have five, we’ve been able to have people focus on those different topical areas.

How do projects get started under the CAR team’s current structure?

It’s a mix of things. Sometimes, just the news demands that we do something. The Census is a really good example of that. The Census Bureau releases data on a very regular basis, and that’s something that we pay very close attention to. We always know what’s coming, and we always are planning well ahead to make sure that we do things around that topic. That’s an example of an event dictating that we do something. Another thing would be the NFL draft, if we’re going to do some analysis around what players have been selected and where they’ve come from, or do an interactive around that, like we did last year. There’s that driver, news and events, and then there’s the driver of things that reporters bring along, or editors bring along, but I would say even in those cases, it’s still very much driven by topics that are in the news, or things that have risen to prominence on the radar screen or the surveillance that reporters and editors are always doing off of their beats.

What enables USA Today to maintain a large-enough data staff to practice CAR across multiple sections?

The decision was made a number of years ago that we would really spread our focus around the entire news operation when it came to developing CAR. I think we’ve known intrinsically that there really aren’t too many beats that journalists cover where CAR cannot play a role. Never mind that many, many journalists cover things where the federal government or a state or local agency would play some kind of role where data gets generated by those agencies and would be of interest for analysis. Even in cases where that’s not the case, there are very often sources of data or the ability to create your own data sets, off of just about any beat that there is. I think that’s something we’ve recognized, for many years, and we’ve really tried to do our best to dig in places that sometimes journalists don’t always dig in. And I think we have to recognize that when you’re talking about a news organization that has one CAR person, then very typically they’re going to focus on things that are generally what you would find a Metro desk or a news desk focusing on. Those tend to be the things that journalists want to hit first: education, budgets, spending, and that sort of thing. But we’ve had the relative luxury of having more than one CAR person or database editor, so we made the decision early on not to focus all of our people on those traditional news section topics, but spread out into the other areas as well.

Do you ever do breaking news, as opposed to the long-term projects?

Certainly, when the Census Bureau dumps a new round of the American Community Survey on the world, that’s a breaking dump of data. And we will take that and turn around stories pretty quick. Up until recently, the Census Bureau was releasing data under embargo, so they would give us two or three days. They’d put the data out, and journalists would be able to get it and have two or three days to analyze it and write some stories, and do some maps, or some interactives. So, we would do it that quick.

But, just in the last while, the Census Bureau has made the decision to stop that whole embargo process, and just release the data. Not only are they doing that, but they’ve decided to release the data at midnight. A couple of weeks ago, they were releasing some new data, and Paul Overberg was at his computer at midnight waiting for it to happen, and I think they had a story up on our Web site within an hour or two. I don’t know if they got it into print that day, or followed up the next day. The NFL draft last year, we expanded an interactive that we had and we built some backend database programming that allowed our NFL desk to enter into a Web form every time another player was drafted, they entered the player’s name, and what school they came from, and what position they played, and what team picked them. It automatically updated our interactive But much more of our effort is on non-breaking news, on projects, or longer-term analysis.

This week, Jack Gillum did a story with one of our education reporters, Greg Toppo, where they got some data on Advanced Placement test taking, and they were able to do some analysis and show that the number of people who were taking AP tests has increased, but even as that number has increased, the percentage of test takers who get a failing grade has also increased. They worked for a couple weeks on obtaining that data, cleaning it up, and running statistical tests on it, doing reporting. Sometimes those are things you have a really hard time doing on deadline. It takes a couple days, or sometimes a couple weeks, to really gather the data and do a good analysis on it.

What do you think of the posting of uncontextualized data centers?

We CAR people, those of us who have been a part of IRE, and going to NICAR conferences for years, we’ve been talking about data for – for me, 15 years, for other people, like the Steve Doigs of the world, probably for 10 years or who knows how many more than that. CAR people have been talking about data for a very long time, and we’ve always talked about data along with the context of the data in the stories that come out of the data. We’ve never really divorced the notion of context from data, because to us, data was never just a means to an end in itself, it was always the driver of a story.

I’ve been to many CAR conferences where people said something along the lines of, “Data analysis doesn’t always give you the story, it gives you the questions to ask that lead to the story. Or it gives you the specifics that help you tell the story.” But I think what happened, and I wouldn’t want to point the finger at Gannett, because I think it’s happened all across journalism in general, is that all of a sudden a lot of non-CAR people discovered data and databases, and said, “Hey, this is great, let’s put these up on our Web site.” In doing so, I think we run the risk sometimes of presenting data without that context, and without the story that goes along with it. It’s one thing, for example, to post the salary of every teacher in the county that you live in. Certainly, there’s the nosy factor, where somebody wants to go and look and see what Mrs. Magillacutty makes as principal of that high school. But I would much rather see five or six or ten years worth of that salary data analyzed, and then presented in a way that tells me what the real story is behind that data as a whole. What are the salaries of first-year teachers, and how have they changed over time? How has the recession impacted what teachers are getting paid? What’s happening to teacher salaries as baby-boomer teachers start to retire? These are all the kinds of questions that I think a smart data analyst is going to ask about that series of numbers, and look upon the mere presentation of them with a bit of a skeptical eye, because it’s very hard to just look at a series of numbers and discern what they mean.

If I were to look up Mrs. Magillacutty’s salary, and find out that she makes $65,000, what does that mean? Is it high? Is it low? Has it changed over time? What year is she in? I think we need to tell the readers more. I’m not saying that there’s no value at all to presenting data on Web sites. I just think we always need to think about, what is the story that we’re telling? There always has to be a story. And that holds true no matter what we’re doing. Whether we’re writing a two-paragraph brief, or analyzing five million records of something, or creating a spectacular interactive out of Django and the Python framework.

What are your thoughts on data-driven applications as being distinct from the data centers?

I think Politifact tells a story. I think that does make it different from just a data center. When I go to Politifact, and I just start poking through the various statements that different people have said, and whether or not they’re “Liar, liar, pants on fire” or whether the truth meter is pegged over into the truth zone, that starts to let me tell a story. See, that’s the great thing about really well-done interactives, is that it lets the reader tell his or her own story. And it lets the reader discover the context. It lets the reader discover more than just a list.

For example, the New York Times did a visualization around Netflix. Now, there’s a certain gee whiz factor to that, because they did a really good job with the mapping, and I think anybody who’s kind of a mapping nerd would look at that with a little bit of “Wow, that’s pretty cool.” But if you get beyond that and you really start to play with it, and I like movies a lot, and I kind of understand because I did CAR related to entertainment for many years, I understand how the movie industry works. Once I got past that gee whiz factor, I started to realize that it actually was telling me some stories.

For example, there were some movies that played very well in urban cores that did not play well in the suburban ring around the core. There were some movies that seemed to do much better in the South, than they did in the North, in the Midwest.. And see, that starts to tell me a story. The neat thing is that I was discovering it as I went. Now, I never did see the story that they did around that, but if I were going to jump off and write a story based on the data, boy, that gives me some great thing to explore. I think there’s great potential and great promise in really, really good data-driven interactives.

We’ve done several at USA Today that I think allow the reader to tell a story. Paul Overberg keeps a database of every solider who’s died in the Iraq and Afghanistan wars, and we have an interactive where you can see pictorially represented, each one of those deaths. Then you can go through, and you can filter and select out people by various demographic characteristics. So if I just wanted to find female soldiers who had been killed, I can do that. And the way the interactive is set up, I get a real visual sense right away of what proportion of soldier deaths are women. And I could do that for anything. White vs. Hispanic vs. black. Or age groups. I can quickly look and find everybody over the age of 50.

Do you see a role for CAR specialists in the future?

There’s certainly a lot of interest right now in people learning how to program in frameworks, whether we’re talking about Django or Ruby on Rails, or even like us, we do a lot with ASP.net. There’s no doubt that we really need to develop that area of expertise in newsrooms to present data interactively in ways that tell stories. But I don’t think we’re going to lose the need for people that can mine data to find the trends, whether or not we’re going to present those trends visually or use them very simply to guide the reporting of a topic. I think we’re going to need both. But I think it’s very hard for one person to specialize very well in everything.

I think it’s very hard for people to be an expert in mapping and an expert in frameworks and an expert in statistics and an expert in data parsing and an expert in writing – there’s a lot there. Right now, I think we have a lot of people who are doing their very best to try to cover all those bases, but people being people, we all gravitate toward different parts of that spectrum, because some of us are more interested in coding, and some of us are more interested in getting public documents and parsing them. The ideal situation is to build teams, where you have people who can specialize in different components and then combine them into the power of a team. For smaller news organizations, there’s a real need for the news organizations to identify and train people to get those skills and to be able to be a part of those teams. The days of having the CAR person in the newsroom, and expecting the CAR person to do all this work, they’re gone. Ten years ago, the amount of data that we had access to and that we were expected to deal with, that was like the drip of a leaky faucet compared to the firehose that’s out there now. We’ve got the Obama administration’s Open Government Directive and every government agency directed to put three high-value data sets up online. We desperately need for more people in newsrooms to get in on this.

Do you see the explosion of data in the modern era as a good thing?

Definitely. I’ve always told people to treat data just like you would treat a source, a human being source. Especially when I teach basic CAR to students, I always tell them when you get a data set, you have to interview that data in the same way you would interview a source. You have to find out if it’s credible, you have to find out what its breadth of knowledge and limitations are, you have to find out what its characteristics and its behavior is. As reporters and editors, we all know that there are some people out there that you can call up, and they’re like an encyclopedia about a topic, and there’s some people you call up and they really don’t have much to offer you. Well, it’s the same way with data. There are some data sets out there that are rich and deep and valuable, and you’re going to go to them again and again, and there are some where you’re like, “Meh, doesn’t really do much for me.” But the only way to discover that is to really interview them and find out what they’ve got, and what they’re really telling you.

Related posts you might enjoy:

• February 20, 2010 -- Data Delver: Chase Davis, California Watch
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• February 13, 2010 -- Data Delver: Mo Tamman, Wall Street Journal
• February 12, 2010 -- Data Delver: William Hartnett, Palm Beach Post
• January 12, 2010 -- Data Delver: Ted Mellnik, Charlotte Observer database editor
• March 28, 2010 -- Data Delver: Mark Schaver, Louisville Courier

This profile of LaFleur is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

LaFleur comes to journalism from a family of engineers, and was looking for a way to use math along with her passion.  She entered computer-assisted reporting when it was still fairly new, remembering the very creation of NICAR at the University of Missouri. Since then, she’s worked at a variety of newspapers, and also helped develop the training program at Investigative Reporters & Editors.

At ProPublica, the work she does is similar to her past jobs — it’s all focused on investigations.  But, she said, there’s a freedom to working outside of the newspaper realm.  There’s more room for experimentation, such as with data visualizations.  And there aren’t as many things that you have to do, just because you’re serving a particular region.  “If I were to go to a newspaper in a state, we would probably build a library of all kinds of databases from that state, just standard things you need to have,” she said.  “But here, we don’t really know what people are going to be covering, so we’re not doing that right now, we’re keeping data as we get it for particular projects.”

With the right attitude, CAR’s just a way of practicing journalism. LaFleur said she doesn’t like to compartmentalize reporting.  “I don’t really look at a database any differently from a bunch of documents,” she said.

And the more fluent general reporters are in skills like Excel and Access, which was the situtation at the Dallas Morning News, the more CAR editors can focus on longer-term projects.

She thought theses skills would be more prevalent across the board in 1995, back when she was IRE training director.  “I said that everyone would be doing this, and there would no longer be a name for the specialty, and I was very, very wrong,” she said.

She’s surprised there aren’t more women in this field at this point, but said that her gender hadn’t provided any benefits or disadvantaged during her career.  As a fun fact, in the earlier years of NICAR, a group of women could go out for dinner at conferences, calling themselves “CAR Chicks.”

Extended transcript

For more of LaFleur’s thoughts on computer-assisted reporting, and how it’s applied at ProPublica, read on.

How did you become interested in computer-assisted reporting?

I was interested in it before the term existed. I wanted to study journalism in college, my family’s all engineers, and they were appalled by the notion of becoming a journalist. So, I basically studied journalism and computer science and math, and later went to grad school at the University of Missouri for journalism, and while I was there, Eliot Jaspin, who’s one of the pioneers of computer-assisted reporting, came to found the Missouri Institute for Computer-Assisted Reporting, which later became the National Institute for Computer-Assisted Reporting. I had the opportunity to work with him, and that’s what got me started.

Where did you go after Missouri?

I ended up working in Washington D.C, because Missouri has the Washington, D.C. program, and I just stayed and worked for some smaller publications there, and then ended up going to work for the San Jose Mercury News a long time ago, before there was any computer-assisted reporting there. I ended up doing a lot of data work on the 1990 Census, and that got me going. I left for two years to develop the On the Road training program for Investigative Reporters and Editors, then I moved back to San Jose. I also worked for the St. Louis Post Dispatch and then Dallas Morning News.

How do see your work at ProPublica as differing from your experience working at newspapers?

The fact that it’s investigative isn’t different, because I also worked on an investigative team. What’s different is that we’re in the process of evolving and experimenting. So we don’t have all those many things we have to keep up with, like at a daily newspaper. If I were to go to a newspaper in a state, we would probably build a library of all kinds of databases from that state, just standard things you need to have. But here, we don’t really know what people are going to be covering, so we’re not doing that right now, we’re keeping data as we get it for particular projects, but everything is so new that it doesn’t make sense to do that. We also have the opportunity to be able to really do cool things like experiment with visualizing data online in a way that, when you’re confined by a corporate structure, you really can’t do. It gives us a lot of flexibility to try things.

What are some of the ways you keep up with where data is being released?

If you cover a beat, you find people that are good sources, and you find data that are good sources.

At ProPublica, how do you keep up with new data being released, when you are looking at a broader range of coverage?

I’m not looking for new data to be released, necessarily. A lot of people here working on stories know about stuff that exists on their beat. In other cases, I just try to find stuff that fits with their beat. Basically, if records are gathered, or documents are gathered, there’s probably a database somewhere.

How do you see your organization as differing from other investigative groups, such as the Center for Public Integrity, or California Watch?

I think we’re probably more similar to California Watch than we are the Center for Public Integrity, which I admire greatly, they do projects only once in a while. We have a pretty large Web staff here, and have made an effort to keep our site fresh, and to post new things all the time. And if something does break on something somebody’s been reporting on, then we’ll post that

What skills or tools are you using most in the newsroom?

It varies depending on the story. I use Excel and Access a lot, I have databases in SQL Server, and I do access those, but that’s just where the databases happen to be. I use statistical software, I use all kinds of other software to clean all the dirty data that we have to deal with. Most people in the newsroom have Excel and Access on their machines, and that’s the standard for the newsroom. But I have other programs like statistical packages and mapping software,

Do you think there will continue to be a role for a CAR specialist in the future?

When I first started IRE training in 1995, I said that everyone would be doing this, and there would no longer be a name for the specialty, and I was very, very wrong. I think it goes in phases. I think it depends on the news organization. Particularly when I was in Dallas, because everyone all the way up the chain really believed this was an important part of anybody’s reporting, we had a managing editor who had won a Pulitzer for a CAR-based story, it was much more integrated into the newsroom than in many other places. It got to the point there where every team pretty much had somebody who was really good, so I didn’t have to worry about helping everybody. I could focus more on longer-term projects, because the education team had somebody who was pretty good and could help them with most of their stuff, the public safety team had somebody who was really good. I think when you do that, and spread it out, it really enforces the power of the tool.

During your various positions , have you spent most of your time in the newsroom, out reporting in the field, or both?

I think reporting is reporting. If it happens to be that your records are in a database, you need to be able to use those. If it happens to be that you have to go talk to people, you have to do that. I don’t think you can compartmentalize journalism that way. I don’t really look at a database any differently from a bunch of documents.

Have you encountered any advantages or challenges being one of the few women in the field?

I’m actually surprised at this point. I don’t know why that is. We actually used to have a “CAR Chicks” dinner at the NICAR conference. When I was first doing this, I was probably one of the first five CAR editors at any newspaper, and there was one woman in Detroit, and there was one in Boulder, but I don’t know what happened. I don’t know what they’re doing now, but they’re not in newspapers.

Related posts you might enjoy:

• February 12, 2010 -- Data Delver: William Hartnett, Palm Beach Post
• March 8, 2010 -- Data Delver: Matt Wynn, Arizona Republic
• March 8, 2010 -- Data Delvers: Ben Welsh & Ken Schwencke, LA Times
• February 20, 2010 -- Data Delver: Chase Davis, California Watch
• February 13, 2010 -- Data Delver: Mo Tamman, Wall Street Journal
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today

I had many questions for Welsh and Schwenke — the men behind these applications. And thankfully, they were happy to chat.

This profile of Welsh and Schwencke is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

Journalist to programmer

Ben Welsh came to the Los Angeles Times through data analysis, learning about the field through NICAR at Missouri, and practicing it at the Center for Public Integrity in Washington.  His first real foray into journalism was working with Chicago’s Carol Marin and Don Moseley while he was a student at DePaul University.  He’s a mostly self-taught programmer, and has many great ideas about how we can use data to enhance journalism.  He openly admits Web development was relatively new for him when he started at the Los Angeles Times.

He first learned about Web development as a way to practice journalism when he heard about from Derek Willis and Aron Pilhofer, now both at the New York Times.  Welsh said he felt they spoke to him because they were connected to the Center for Public Integrity, where he worked, and were speaking at NICAR, the organization where he first learned data analysis skills.

“I was positioned very close to them, where I could get the message clearly,” said Welsh.  “I didn’t have to go seek out the message, the message was nearby.  So I said to myself, ‘Well, that seems pretty cool.’”

But thinking something is cool doesn’t mean you know how to do it.  Welsh had been teaching himself Web development on weekends.  And the first project, California’s War Dead, at the Los Angeles Times was a true adventure.  “At that point, I didn’t even really know how Apache worked,” Welsh said, “and we just faked it until we made it. ‘Yeah, we can hit the deadline.  Yeah, we can get the Web site up.’ And version one wasn’t perfect, but we shipped it, and it got out, it went okay, it didn’t crash too bad, and I learned a lot.”

From programmer to journalist

Ken Schwencke thought he was going into computer programming when he entered the University of Florida.  But he shifted to journalism upon realizing that the math requirements were “soul-crushing.”  That was where he met Mindy McAdams, a journalism professor who he said opened up his eyes to the possibilities of combining the two fields.  He was interning at the Times before he officially graduated, taking classes from a distance.  His internship was extended, and eventually converted into a full-time job.

Journalist + Developer = Journalist/Developer

Welsh explained his job as the combination of two fields.  “The technical work is pretty much identical to the typical web developer, but the approach to the data, and the presentation of it, requires the same artistic and analytical skills as being a reporter,” he said.

Projects range from ways to allow users to explore databases to housing visual projects, often Flash or video, requiring intense resources outside of the paper’s CMS structure. Ideas for projects fall into three general categories, although these aren’t the only ways ideas are generated.

• Paper to Web.  The paper is working on a story, and an editor wants to blow it up on the Web.  Example: Mexico Under Siege
• Not possible on paper, dream comes alive on Web.  Someone has an idea for something so dynamic and non-linear it demands the Web.  Example: Doug Smith wanted to do something with mapping neighborhoods, and allowing readers to contribute.  Mapping L.A.
• Editor gives general categories.  Former Web editor Meredith Artley would tell Welsh and Schwencke — whom she called the “geek squad,” she wanted to see pieces in certain general categories.  It was up to them to come up with ideas for the specifics.

The deadline’s the thing!

Welsh points out that for all the intricacies of using programming for analysis, presentation and deployment, it’s essential to remember that you’re in a deadline environment.  Expectations are reasonable, and others trust the team’s assessment of how long something will take.  But like any job in journalism, it needs to get out on time.

Long projects are given long deadlines, but sometimes applications go along with breaking news, and must be done that day.  “Hazy deadlines are common,” said Welsh.  “But at the same time, so is an, ‘Oh my God, we’ve got to get something done before tomorrow’s paper’ attitude.  It can often come on and you just get all hands on deck to get something done quickly. ”

When that happens, you’ve just got to get it done, Welsh said.  Even so, there have to be limits, especially when you understand how long something takes, and others may not.  “in the long term, you can’t be the person who always says yes, because you’ll kill yourself,” said Welsh.

It’s not a side project anymore

For a lot of journalists who go into programming, they get their practice creating side projects.  But the traffic one gets at a personal site is vastly different from the traffic your sever endures when you’re featured on the latimes.com home page.  “Before, when I was building apps largely on my own time, if my data wasn’t perfect it was usually good enough,” said Schewencke.  “Now the bar is much higher. I also now need to worry about things like scalability — am I hitting the database an unnecessary number of times? Will my app be able to handle traffic from the top spot on the front page, or will it cause our servers to melt down?”

Benefit of a small team

Because it’s pretty much just Welsh and Schwencke, communicating is fairly simple.  “We sit right next to each other, so collaboration is as easy as me yelling until he takes his headphones off,” said Schwencke.  They use GitHub to manage different versions of their applications, and usually don’t work on the same project at the same time.  That means if Welsh is deep into a program, and something new comes up, it’s usually Schwencke’s responsibility. Or vice versa.

The peril of meetings

Both Welsh and Schwencke mentioned that one of their favorite parts about the job is being able to try out interesting new ideas with smart and dedicated people.  The flip side is that sort of work requires a lot of organization, and a lot of meetings.  “As most organizations do, sometimes we can fall prey to designing by committee, or getting sucked into a lot of meetings…The conversations and feedback can be really fruitful, but sometimes it makes me want to stab myself in the face with the nearest pen. Small price to pay to do something I love for a large audience.”

Extended transcripts

For more on Schwencke and Welsh’s stories, and their experiences at the Los Angeles Times, keep reading. Selected responses from my email with Schwencke, and below that, my phone conversation with Welsh, are below.

Ken Schwenke’s email responses

How did your experience in programming grow?  Did you take formal classes, learn everything on the job? Any people/books/Web sites that were particularly helpful?

I started programming when I was young — the end of middle school maybe? Certainly by my freshman year of high school. I remember riding my bike down the street to a now-closed book store to pick up such gems as C For Dummies. I had an old computer at home that wasn’t hooked up to the Internet, so I would go upstairs and carefully key in examples and try to figure out how the hell things like linked lists and pointers worked. There’s still a stack full of books ranging from assembly programming to Programming Windows to C algorithms sitting at my house in Florida. I aced AP Computer Science in high school and then switched to journalism in college largely because I thought the math requirements for computer science were soul-crushing — and hell, I was a pretty good writer.

As for helpful books and sites….when I got back into programming (this time with an eye towards Web and data stuff — credit Mindy McAdams for getting me back on track), I voraciously read everything on programming.reddit.com and eventually news.ycombinator.com. I originally started back up in Perl (I’d tinkered with it before I stopped programming, and I knew it was good for scraping and mangling text) and found Programming Perl to be a really useful book, though I know people think Perl is passe now.

What was your general path to the LA Times after you finished school?

It was a pretty direct route. I technically started interning at the Times before I graduated. I had a single credit left to handle in GIS before I could get my degree, and I hammered it out remotely with a professor of mine over the summer. Mindy knew the Times was looking for a programmer, so she told me and I contacted [latimes.com managing editor] Dan Gaines and went through the usual round of interviews with him and Ben. It was actually my first internship. Before that I had been working at UF’s school paper, the Alligator, in various roles (including ME/Online), but had never worked in a real newsroom until then.

What do you find to be the hardest part about the types of applications you build?

Making sure the data is correct and in-tact is key. Before, when I was building apps largely on my own time, if my data wasn’t perfect it was usually good enough. Now the bar is much higher. I also now need to worry about things like scalability — am I hitting the database an unnecessary number of times? Will my app be able to handle traffic from the top spot on the front page, or will it cause our servers to melt down?

What tools do you use to facilitate collaboration between yourself and Ben?  Some kind of versioning?

We use Git and GitHub for all of our version control and code. It’s great. We also deploy using a Fabric script, which automates updating the code on our live site. I never knew how much of a necessity automated deployment was, but trust me…I know now. We sit right next to each other, so collaboration is as easy as me yelling until he takes his headphones off.

How do you delegate/split the work between the two of you?

In Django, each app we write (for example, the Homicide Report is one app, Wardead is another, Mapping LA is another) is largely separate from the rest of the code base. Sometimes we pull in data from other apps (like displaying neighborhood boundaries in the Homicide Report), but we’re usually never working on the same app at the same time. Basically, if one of us is working on a big project when another project idea comes up, the other more-free one of us gets to handle it. It seems to work so far, and it definitely keeps us busy. But we’re always bouncing ideas off of each other and getting each others’ input.

What is it you enjoy the most about your job?  What frustrates you the most?

I really like the people I work with. Everyone is very intelligent and willing to try out new things, which is wonderful. It’s interesting work, I’m usually presented with new challenges every day and I get to try my hand at a lot of different things. I spent a few weeks delving into the OpenLayers codebase to re-write their clustering algorithm for the HR main map, and I even landed a small patch in the Django codebase.

As most organizations do, sometimes we can fall prey to designing by committee, or getting sucked into a lot of meetings. It’s (sometimes) a necessary evil. The conversations and feedback can be really fruitful, but sometimes it makes me want to stab myself in the face with the nearest pen. Small price to pay to do something I love for a large audience.

Do you prefer working on breaking news data-driven apps, or longer projects over time?

Since I’ve been here, I’ve yet to work on a breaking news type of app, although I’ve got one coming up soon that should be a lot of fun. There’s a definite difference between putting up something that’s a long-term resource like HR or Mapping LA and hitting a news hook with an app that explains something better than a text story can. So I’m excited to get to try my hand at it.

Extended transcript from Ben Welsh’s interview

Explain your position at the Times.

I’ve been here a little more than two years now, I started in December 2007, the title I was hired under is the position I still hold, which is Database Producer. We now do have a second person who I work with, his name is Ken Schwencke.  It’s essentially the same stuff that Derek [Willis] and Matt [Waite] and other people do, pretty much web development, except for news projects.  The technical work is pretty much identical to the typical web developer, but the approach to the data, and the presentation of it, requires the same artistic and analytical skills as being a reporter.  How you decide what to do, and how you do it, are shaped a lot by the traditions of newspapers.  But the technical work of actually achieving it is pretty much the same as most web developers who work with the same technology.

I spend all day sitting in front of a computer writing code that will, in most cases, power a public news Web site.  We don’t focus much on general CMS stuff that most developers at newspapers had formerly worked on.  It tends to be news applications that are not as tied to the central news engine of the content that gets shoveled through the newspaper everyday.  It’s things that are built for projects, whether they be multimedia- or data-focused, or things that are online only and displayed in a newish way. In our case, it’s a mix.  Just this last week, Ken did most of the work, we put out an update of our homicide blog, which in the past had been a TypePad-powered blog, but is now a Django-powered blog and database, so it has the look and feel so it’s more like a database, but also still has the blog features.  We do other projects that are strictly multimedia that require their own CMS, and careful mounting on the Web.  We’ve got our “Mexico Under Siege” project, which is very Flash-cented.  And we’ve done a couple of other Flash-centered multimedia things – “South L.A.,” “Alabama Homeboys,” and “Obama 100 Days,” where the centerpiece of the presentation is a Flash application with video or slideshows and other rich media that I can take no credit for.  The Web developer’s job is to get it all wired up on the Web in a way that’s nicer than our internal CMS can allow, but also have some of the features that we want, like user comments. It ranges from developing these data apps to multimedia apps, and there’s points in between.

You partner up with different people across the newsroom depending on which kind of thing you’re making.  There’s different modes here. One is, how do we take data analysis, investigative or research wise, that the old-school CAR team has done, and bring it alive on the Web.  “Mapping LA” might be an example of that.  Another is how do we take topics the paper already covers and turn them into these engines online, and that’s more like what we’ve done for the obituaries of soldiers – California’s War Dead where you take something the paper already does, and you sort of Web it up, and make it this new corner of the Web site.  And there’s the multimedia project, where you sort of help enable the very artistic Flash, video and multimedia people, and get their stuff online in a way that is a little bit more flexible than the internal CMS allows.

How much communication do you have with the rest of the newsroom?

Ken and I are our own team in the newsroom. We sit with the graphics department on the second floor, and we sit near Doug Smith’s CAR team, of three people who do research and data analysis in the old-school way, and we have a Metro editor we pair with a lot who sits there with us, too.  And the six or seven of us pretty much manage the whole production cycle. The servers that we host the projects on are Tribune servers, but we administer them, and we manage it all.  Besides making sure it’s plugged in and setting it up in the first place, we make sure everything is running and can handle the whole technical deal. Our interaction with other Web developers within the Tribune company is pretty small. We’re working to improve that, but the projects that we do are directed by the newsroom, and done entirely by the newsroom.  In the past, we haven’t always done that. With the Homicide Report, the generation before what just got released, was a partnership between Tribune Interactive and the people on editorial, but as our skills have increased, we’ve begun doing more and more ourselves.

Are you given assignments? Do you have a hand in proposing ideas for the various projects?

It varies.  In the “Mexico Under Siege” project, an order comes down from the managing editor of the whole paper, this was more than a year ago, in the fall of ’08, that in the next 12 months, this is going to be one of the big stories the paper covers.  We really want to blow it out.  On the print side, that means tons and tons of front-page stories, that means a lot of reporters, that means it’s probably going to be their Pulitzer entry, this is something they’re going to put their resources behind.  In this case, they said, “And we want to blow it up on the Web.”

So, it’s like you become yet another weapon in the paper’s arsenal when the people at the very top decide it’s “time to charge” on a certain story.  From the very top, there was an order that everyone needs to come up with something for this topic, and you guys are part of that effort.  In that case, the directive comes from the top, that this is the topic, and then the Web producers all go and conference with the editors on the project and the reporters and the photographers, and get a sense of what all the assets are, and what the theme of the presentation should be. And then we all go into a cave, and storm up what it’s going to be. In that case, I deserve, I think, very little credit. Besides building the nuts and bolts of a lot of the Web site, I think a lot of the creativity for how it works and how it looks belongs to other people who work on the Web project.  Of course, the content itself belongs to the reporters who provide all that.  That’s an example of one case, top down, where we’re going to do this.

A different example would be the “Mapping L.A.” project, which is where we mapped these neighborhoods in Los Angeles, and we had users submit their own maps.  That was an example where Doug Smith had, for decades, dreamed of doing something about mapping L.A. neighbors, but had never really made it happen.  But the stars aligned where we had the technology to do it, thanks to some of this new Web development, I had the time to do it, and we sold the concept. That’s where this interdepartmental team I work with sold the concept up the chain of command.  The “California War Dead” site is another case where that happened, the paper wanted to do something for Memorial Day, we knew there was an appetite for it, so it was pitched by our team.

There’s other cases where the Web editor we had for a long time, until recently, Meredith Artley, had this attitude where she was like, “Alright, geek squad, I don’t know exactly what you should do, but I know that there’s certain topic areas I want you to focus on. I want something about schools, I want something about crime.” She came up with a short list of two or three things she wanted apps on, or she wanted something innovative or different on for the Web site.  Then, it was kind of up to us to figure that out and try to wow her, or do the right thing. So the direction in that case was very vague.  Also, sometimes there’s a story coming down the pipeline, and they want a map, or a sortable list, or a sidebar toy, and we do those every once in a while, too.

What timeframe do you have for these various projects?

We’ve done a lot of things in one day, particularly little sidebars, it varies.  We’ve done apps in one day, we did this app for the Metrolink train crash. We had a commuter train crash out here about a year and a half ago, and it killed about 25 people. We got an app up within a day that had, when the news broke, a list of all the fatalities, and then these profile pages on all of them, where we had photos and data and comments. And that was an example where a big story happened, I pitched the idea, and we just ran.  So, that was a case where because it was big news, we just decided it was time to do something.

On larger projects, the deadline is always in the future and sort of vague, like any classic newspaper project, where the reporters are given some unspecified length of time to go get the story. Because the newspaper managers, especially at big papers like this one, are accustomed to thinking about projects in that way, they tend to apply the same thinking they have toward print projects toward Web projects.  Hazy deadlines are common.  But at the same time, so is, “Oh my God, we’ve got to get something done before tomorrow’s paper” attitude.  It can often come on and you just get all hands on deck to get something done quickly.  If you think about how assigning editors or print editors approach their job, it’s this variation between managing long-term things, and pushing superhard on short-term things.  The more you do them, the more you learn lessons and approach the Web thing a little different than you would a print thing, because they’re not the same.

Do you find the expectations from management to be reasonable?

Yeah. There is also the acknowledgment that the technical skills we are working with are things the print people don’t totally have a mastery of, so if you give them your honest appraisal of how much work you think it’s going to be, as long as you’ve been consistent about doing that accurately in the past, you have pretty good credibility.  Part of working at a newspaper is you’ve got to be able to deliver on deadline, and if you fail to distinguish yourself as a person who can deliver on deadline, you may lose some points in the print world, because that’s how print people judge each other, because that’s the way things have been run in that industry.  Especially early on, if you’re trying to establish the Web stuff we’re doing as something the paper should invest in and do, hitting deadlines is a good way to build your credibility.  But in the long term, you can’t be the person who always says yes, because you’ll kill yourself.

What has the audience response to your work been?  Do people participate in user-generated content-based (UGC) projects?

We haven’t been super creative about the UGC we do, although I think we do more than most data apps. The UGC we do is pretty limited to comment rolls. I think because we’ve done a lot of lists of casualties and fatalities in train crashes, now the Homicide Report, I think in those contexts you don’t have to police things as much, because I think the people who use the site have a sense of its gravity, and are less likely to engage in spam and flame wars, because who’s going to do that on a page for a dead soldier?  Because of the content of what we’ve done, I think we’ve had an easier and much better time with comments. The comments we get on War Dead are just unbelievable.

We always make a point of getting the post up within hours after the release comes out, because that night when everyone who knows the person is searching them on Google because word’s got out, we’re one of the top results. So you get very heartfelt comments from people who knew the soldiers, and for me personally, that’s one of the most rewarding things about the work in that sense, just because if you’ve made something that people feel so comfortable sharing their private and emotional information on, you feel like you must’ve done something right, even if you’re not sure exactly what it was. I know it’s very easy to trash user comments, and a lot of people like to, but frankly, I think I’ve had a very good experience with our applications. We’ve received things that have really added to the value of the site.

How is the general traffic? Are people using the applications?

Yes.  We have a general traffic level that we get, and then we tend to get very large spikes when we release something, or when something gets a lot of home page attention.  A few of the apps are doing increasingly well with search engine keywords, like it’ll be our goal to be the top search result for all the people we have in different databases.

How did you get started in programming and journalism?

My first journalism job was in Chicago.  I was a student at DePaul University, and I got a gig working as an intern for two great people there – Carol Marin and Don Moseley — who spent most of their careers as television journalists. I just worked as their footman/assistant.  For example, Carol was writing a project this week about Topic X. She wants you to research some information.  Or, we need to set up some interviews, or figure out who to interview, At the time, they were doing some long-form documentaries for CNN and elsewhere, so I would do some research and cutwork.  It was a great experience for me. Carol and Don are very warm people who are good teachers, but they are also excellent about what they do.  It was formative for me in that even though I was in college, and taking journalism classes, that was the moment when I realized, “Hey, I can do this!”  It’s not just something you heard about in class, but you actually do it.  You feel like it’s attainable, and something you like doing.

I graduated from college, and I didn’t really see a journalism job that was great, or maybe I just wasn’t pushy or assertive enough, or didn’t go knock on enough doors, or just didn’t make it happen, but I just didn’t see the opportunity.  So I decided to go to graduate school at the University of Missouri in Columbia.  With roughly the intention, but not totally the realization of what it would entail, I decided I wanted to work with IRE and NICAR while I was there.  I’d heard of computer-assisted reporting before, it sounded interesting. I was interested primarily as a method or a discipline for doing quality investigative reporting, which is what most interested me about journalism.  I had done a small amount of stuff like that with Carol and Don.  I remember we did a story about the city of Cicero and some expenses given to them by a contractor and I had to file a public records request, fight the city of Cicero to make sure they handed it over, and then do a rudimentary spreadsheet analysis of what they sent back, which ultimately resulted in a segment on WMAQ that Carol did.  That was an example where I was like, “Wow, you can do some really cool stuff if you know the tricks of the investigative trade.”

I went to Missouri, but I didn’t see the path I’ve ultimately taken to where I am right now.  I had a great time at Mizzou.  I had a great classmate there named Brian Hamman, now at the New York Times.  I remember seeing Brian and Chase [Davis] and seeing how invested they both were in the Web, even at that time, which would have been maybe five years ago now.  They were much more committed to the Web as a career path than I was.  I remember seeing that, and it was an eye-opening experience, seeing that there were people out there that looked at things that way.  Then, there was an opportunity that opened up to be a graduate assistant at NICAR, and I think Brian might have helped me get the gig. I worked on learning some basic SQL and computer programming to help NICAR do some farmed out investigative work for TV station X or whatever.  I remember doing that, and thinking it was really cool that you could work on a cool story and you could get the job to work on a cool story because you had the technical skills, where if you didn’t have the technical skills, good luck.  It was clear, even from the analysis world, that having the technical skills was your foot in the door.

To me, the combination of seeing people like Brian and Chase, and personally tasting how you would have this opportunity if you would just learn the skills, was a career-changing combination.  And then having the time to reflect on it, because you’re in graduate school.  You’re not busy working all the time like a lot of people had to do it.  I was lucky enough that I didn’t have to work a full-time job while doing graduate school, which is something I’m very grateful for.

After graduate school, I got a job doing that kind of analysis at the Center for Public Integrity in DC. And it was like, “Wow, you can even get a job.”  I worked there for about two years, and was hired by Daniel Lathrop. Agustin Armendariz was there, too. He’s now at California Watch.  And Helena Bengtsson was there, who’s now back home in Sweden at Swedish Public Television, and John Perry, who’s now the CAR guy at the Atlanta Journal-Constitution.  So, there was a period for about a year and a half where we were the data cave of nerds who did analysis for big projects there. We did one on congressional travel, on trips and junkets members take.  We did some projects on lobbying numbers, and telecom policy.  That was where I spent about two years doing computer programming, mostly SQL.  I had the opportunity to work on investigative projects, but also the opportunity to cut my teeth and learn a lot of computer programming along the way. I learned a lot of Perl from John Perry there, who’s a great, great guy, and was super generous with his expertise.  I had the opportunity to close the door, and read the manual, and figure a lot of stuff out.

Around 2006 or 2007, after I started getting comfortable with a lot of the basics of scripting and programming, that’s when Derek and Aron and these guys started really advocating within the NICAR world, which I now had been traveling in since graduate school, these guys started advocating that learning more and more Web development skills was a good idea.  Those guys, who had come up doing analysis with campaign finance and other stuff made the transition to being more Web developers.  I would give them all the credit as far as advocating that, and really helping people see that as a path. Because both Derek and Aron had worked for the Center for Public Integrity, where I had, and they were both active in NICAR, where I was a graduate student, they had a lot of credibility with me. As far as social networks go, I was positioned very close to them, where I could get the message clearly. I didn’t have to go seek out the message, the message was nearby.  So I said to myself, “Well, that seems pretty cool.”

I started teaching myself those skills, just on weekends.  I never really did a whole lot of Web development at the Center, but had been learning different things, when the L.A. Times called, and they knew they wanted somebody to do data stuff for their Web site, but they didn’t really know exactly what they wanted them to do, Caspio was still considered a good option at that time. They were print people.  They kind of knew that they wanted to do more things, and they wanted to do original Web development in the newsroom, but they didn’t know exactly what.  Somehow, I talked my way into the job, even though at that point, I had never really built a serious functioning Web site, particularly one that takes as much traffic as the L.A. Times does.  But I had the desire to try it, and was able to talk to them in the language of news to say that we could straddle this boundary together.

Then, I came to the LA Times and pretty quickly we worked with Doug Smith, who I would give a lot of credit to for trying to find ways to identify projects in the print world that we could latch on to, or take hostage, to try to turn into a Web effort. The first big opportunity we really had was we did some election data stuff with primaries and caucuses in ’08.  The first big opportunity was the War Dead piece in the end of that year, where we just said, “By the way, we’re going to build a Web site.”  We got the editor to sign off, and a little bit of money to put up some servers, and at that point, I didn’t even really know how Apache worked, and we just faked it until we made it. “Yeah, we can hit the deadline.  Yeah, we can get the Web site up.” And version one wasn’t perfect, but we shipped it, and it got out, it went okay, it didn’t crash too bad, and I learned a lot.

We’ve just gradually been doing more and more things like that, and now we have a second person doing development.  I’ve learned a lot about being a systems administrator in the last year and a half. Things I never would have thought I’d know in my entire life from journalism school.  I’m now pretty much spending most of my time doing Web development. A lot of times doing Web development for projects that have an investigative or data analysis bent to them, but really spending the majority of my time thinking about how it’s going to be presented to the public, in a publishing kind of sense.  I think that is a real fault line in the whole CAR community.  This distinction between presentation and analysis.  I’m someone who was doing analysis full-time, but who took the leap into doing a lot more presentation work.

It was a leap that was good for me, because it was a career opportunity for me, I was a young person, and I could make more money and work at a bigger media organization, and live in Los Angeles, which I would recommend to anyone.  For me, it came at a point in my career where it was very advantageous.  If you already have the great analysis job, it may not seem as personally advantageous to you.  That’s just doing the calculation of self-interest. There’s also the real objective of “Are we creating better news products that are in greater service to the public interest?”  I’ve crossed over in a sense.  Part of me is like, “Well, I feel really passionate about great investigative journalism, and I want to keep doing it.” But this other stuff is really challenging and fun and you get to do things that are still rewarding and good, too.  It’s a move that’s been mostly comfortable, but where you’re ultimately going to end up on that spectrum, I don’t know.

Related posts you might enjoy:

• March 8, 2010 -- Data Delver: Matt Wynn, Arizona Republic
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• January 31, 2010 -- Data Delver: Matt Waite, St. Petersburg Times
• January 29, 2010 -- Data Delver: Perry Swanson, The Gazette
• January 12, 2010 -- Data Delver: Ted Mellnik, Charlotte Observer database editor
• March 28, 2010 -- Data Delver: Mark Schaver, Louisville Courier

The CAR world, as I see it, has two different paths you can go down: continue to use data for reporting stories, or apply those skills to web development and presenting data. The latter splits into front-end and back-end work as well.

If you’re indecisive like me, the best case scenario is to be able to use CAR for both reporting and building data apps, or at least, exercise careful control over where you direct your skills in different situations. That requires you be an excellent reporter and developer — it’s a tall order.  But one that California Watch’s Chase Davis meets with passion and skill.


This profile of Davis is a part of my continuing series I’m calling “Data Delvers,” where I pass on summaries, quotes and audio clips from conversations with journalists using technology to find, analyze and convey data-driven stories and/or projects to the modern audience.

From AOL games to data journalism

While many CAR lovers come to the field via journalism, and then teach themselves the coding part, Davis was a coder before he knew he wanted to be a reporter.  He was adding extra levels and characters to AOL games with QBasic while he was in junior high.  Then, he fell into reporting in college and discovered he could combine the two to pursue his passion of investigative reporting.  “It offers one of the last new frontiers or opportunities, I think, to find stories that absolutely no one is finding, whether that be reporters, or regulators, or anybody else,” said Davis.

Davis has worked at a variety of news organizations from Boston to Houston, sometimes working as a reporter, sometimes as a CAR specialist, sometimes as a developer, but most often employing many of those skills at once.

Passionately pursuing investigative reporting

Davis uses his technical skills to find stories as an investigative reporter at California Watch.  He shares the Money and Politics beat with Lance Williams, who broke the Barry Bonds steroid story, has more of a historical perspective, said Davis, so together their strengths play off of each other. Davis meets sources in the field and does traditional reporting, but is also able to bring together database applications when needed for packages.

Navigating a spreadsheet should be a skill most reporters have under their belt, Davis said.  But there’s still a place for a specialist working with skills that are “a little bit trickier.”  For example, at California Watch, they are starting to use a technique called “machine learning,” which Davis said means “writing a program that can read documents, and find patterns in those documents that people wouldn’t otherwise see.”

The freedom to experiment

The openness to new ideas for presentation and reporting at California Watch is one of Davis’ favorite parts of his relatively new job. “There’s really no limit, no one’s putting handcuffs on us saying, ‘This is how it’s always been done,’ or any other barriers to any new idea that we have.  So we are pretty much free to do whatever we think could be cool, which is nice.”

That experimentation has included the crowdsourcing of a database.  And there’s more to come.  “We have really big plans for some things down the line that are going to be really exciting, and we’ll be pushing the boundaries of where some of this can go,” said Davis.

Users care about interacting with data

Davis said he’s found people to be extremely interested in this type of work. That’s partially because the stories are so new,and partially because of voyeuristic curiosity.  But, he thinks what people really like are data-driven applications that people can truly engage with.  “They like being able to go in and not just look up what their neighbor is making, but also being able to understand better context locally on any given issue that they’re interested in, by virtue of the database,” said Davis.

Take control of your skills

Davis said having CAR skills of any sort, but especially you combine it with programming, is an extremely powerful and in-demand combination.  He gets calls every few weeks from editors and managers looking to fill CAR-related positions.

He said starting one’s career at a smaller newspaper, and trying to integrate CAR, can be difficult, and has seen that frustration play out among his colleagues when they don’t have ample software, time or support.

But there are many opportunities where people do understand the skill set, and if you have the skills, then it’s up to you how you best want to use them.

“I think the trick is to get to know the people in the community,” said Davis.  “There are so few people who know how to do this stuff, and so many openings, and so much demand, that this is really the one position in journalism, at least that I can think of right now, where they can’t fill the jobs fast enough.”

Extended transcript

Read on to learn about a recent California Watch experiment tapping into community involvement, and more of Davis’ thoughts on the future of CAR.


What is it you do at California Watch?

We are a team of six reporters,three editors, and two multimedia producers, we’re going to hire a couple more reporters, I think, in the next few months. We’re split into two groups. We have the main office down in Berkeley, and I’m in the bureau up here in Sacramento. I cover money and politics issues, along with Lance Williams who came from the San Francisco Chronicle. He is the guy who broke the Barry Bonds steroid story, and he has been doing California politics coverage longer than I’ve been alive. He also has more of a historical perspective, and I have more of the technical skills, so we’re able to play off of each other in that regard.

On my typical day, I go into the office and I report, for the most part. My job is to be an investigative reporter here more so than a developer or CAR person. We have a guy on staff who is amazing with that kind of stuff, his name is Agustin Armendariz. He comes from the San Diego Union Tribune, and used to work for the Center for Public Integrity for a long time. He does sort of the day-to-day heavy lifting on data stuff. I just got back half an hour ago from a lunch with a source, just doing what the standard investigative reporter does. I try to incorporate data analysis into the stories I’m working on. I’ve got something coming out in a couple of weeks that’ll have some interactive databases and applications going with it. We had a story that I did a couple of weeks ago that had the same, and so we sort of try to incorporate it.


What are some of the differences between the work you are doing at California Watch, and what you did in Des Moines?

Des Moines, actually not much. I was also a reporter there for the most part, I was also the CAR specialist there, I suppose. The charge there was more writing stories than it was doing analysis work for others. The difference here, I suppose, other than the structure of the organization, is that I’m on the beat of money and politics now, whereas in Des Moines I was just on the investigative desk, I worked on environment stuff, and some other things.


Can you take me through your career and some of the other positions you’ve had, and tell me how you got started?

I went to Mizzou, and did internships every summer that I was there, it ended up being six internships, and one was in the fall or the winter. I started off at a small weekly in Minneapolis, then I went to Iowa, then to Omaha, St. Pete briefly for a little special thing we worked out over winter break, then to Milwaukee, and then Boston. About halfway through that time I started working at NICAR in the database library, just kind of on a lark, I didn’t know where that would actually take me. I knew how to program in a handful of languages, and I had some experience with databases, and so they let me come on there, first as a volunteer, and then as a paid staff member. So I worked with them for a couple years, sort of doing data analysis and cleanup.  And then, the Boston thing was an extended internship where I actually went to Boston for about five months doing night cops reporting for the most part. But then after that, I stayed on remotely for another six months or so, just sort of helping out with the various stories with the Romney campaign, when he was gearing up for his presidential run, doing sort of data analysis, and things like that, for the political reporters. I got hired in Houston, worked at the Chronicle for a couple of years, on their investigative desk as their CAR specialist, and then from Houston, I went to Des Moines for what ended up being a very short amount of time – about nine months—before this offer came up, and now I’m here.


What drew you to CAR in the first place?

I come to CAR from a weird background in that I knew how to do computer programming and all that before I knew I wanted to be a reporter. I learned how to program in fifth or sixth grade. I was in a summer school class, and then spent my junior high years sequestered in my basement modifying QBasic games that I downloaded off of AOL, adding new characters and new levels. I thought I wanted to do computer stuff before I wanted to be a reporter, but then I fell into the reporting thing, and then I kind of discovered about halfway through college that there was a way to combine those two.

It was attractive to me in large part because it was a window, or gateway, into investigative reporting, which is what I’d been interested in in journalism. Also, because it offers one of the last new frontiers or opportunities, I think, to find stories that absolutely no one is finding, whether that be reporters, or regulators, or anybody else. Most people don’t have the technical expertise to sort of pursue a lot of these stories, and so it’s really just fertile ground and stuff that nobody has ever looked at – you can find new stuff every day.


Where do you see the future of CAR?

I think everyone’s going to have to be required to do some of it. If you can’t navigate a spreadsheet, at some point that’s going to become a hindrance to your career as a reporter, especially as more public records are put into electronic form. But I think that there’s always going to be a role for the specialist in that, because not everybody is going to know how to program, not everybody is going to know how to do some of the higher level stuff, nor should everybody have to know how to do that. It really only serves you in certain situations. I think that everyone’s going to have to learn more about how to do analysis, but still, there’s going to be a role for people who know an exceptional amount, and can do some of the things that are a little bit trickier.


What are some of the skills that you consider to be trickier?

A lot of schools are doing a good job now teaching the basics of Access and Excel. I taught a CAR class, for example, at the University of Houston, where we walked people through just how to do basic stuff in Access and Excel. If you were confronted with a database or a spreadsheet, you’d at least know what to do with it. Mapping used to be the classic example of the next step, but not a lot of people are going to have to know how to use that. But it’s going to be good to have someone in the newsroom who can, if the occasion arises.

Anything related to programming – it’s going to be good for a reporter to pick up at least some bit of programming, but I don’t think that’s feasible for all reporters to sort of have a background in that. Building online applications will still probably require some specialization. Even complicated analysis. If you look at something like Matt Waite’s Wetlands, you can’t just expect an average reporter to do that, with satellite analysis and all that.

We’re starting to work now on stuff here that relates to machine learning. It’s basically writing a program that can read documents, and find patterns in those documents that people wouldn’t otherwise see. People shouldn’t be expected to have to do those things to be able to report, so long as you have someone around who specializes in it, you can keep sort of pushing that and keep finding your stories that way.


How interested have you found readers to be in this type of work?

Extremely. I think there’s two sides to that. One is the novelty of the stories that come out of this and the fact that it’s something the readers haven’t read before. It’s not just reading the same-old, same-old campaign finance report story, so-and-so raised the most whatever. You’re revealing new patterns that people haven’t seen and would be interested in.

The other is the online component that’s come along in the last few years of posting these databases online, and not just data ghetto-style databases, as Matt [Waite] would call them, but more like real interactive web applications, like Politifact. There’s the voyeuristic aspect on the one hand, where people just want to see what their neighbors make, if you’ve got a neighbor that’s a state employee, and you’ve got a state salary database up, and that’s for the cheap, cotton candy-style hit. You go and you look it up once, and you’ve satisfied your voyeuristic desires, and now you can just be on your way. But then there’s stuff that Matt builds, and we build on our site, and those are designed for more engagement, and those are the types of things I think the readers really like, being able to go in and not just look up what their neighbor is making, but also being able to understand better context locally on any given issue that they’re interested in, by virtue of the database.


How is California Watch utilizing the Web to convey information? How do the reporters integrate with the Web team?

Well, we have two multimedia producers, and we have Augie, the CAR guy, and we have me. Our Web site, we actually contracted out for that. But, the plan is to use it in whatever creative ways we possibly can. The nice thing about working here is there’s really no limit, no one’s putting handcuffs on us saying, “This is how it’s always been done,” or any other barriers to any new idea that we have. So we are pretty much free to do whatever we think could be cool, which is nice. So, we started off with a data ghetto-type page, with a bunch of searchable databases in different topic areas, which people have used actually quite a bit. We tried a little crowdsourcing experiment with an online database earlier on this month that’s gotten results, and we have really big plans for some things down the line that are going to be really exciting, and we’ll be pushing the boundaries of where some of this can go.


A few weeks ago, California Watch staff went out to coffee shops to hear from people in the community. How did that go?

I kind of had a weird day that day. I went to the coffee shop for an hour and a half, two hours, that morning. It was a coffee shop where all of my sources go anyway, so I ran into a lot of them, and they were familiar with the idea. One person made it a point to sort of come and talk to me, he was a political operative here who knew we were doing this, so he made it a point to come up and say hi. But most of the people I ran into were people I already knew. But then I also went down that same day to a public library in Lodi, Calif., which is about half an hour, forty-five minutes south of Sacramento, and did a talk for both the local newspaper and people from the community who wanted to know what we were about, and just generally know about politics issues. At that one, other than the local newspaper, we probably had maybe a half-dozen community members, including a local city councilman and some other folks who showed up, and had a real interest in what we were doing.


Do you have any advice on getting into the field, or deciding what part of computer-assisted reporting to specialize in?

You can do whatever you want with the skills. In my day job, I’ve taken more of a reporting angle, rather than wanting to create database applications, I use the skills as a reporting tool to find new stories. In my night job, I build Web sites with Matt, so it’s sort of like I have both sides. In general, I’ve looked at this more as a way to enhance my reporting, and to find stories that other people can’t. But other people have chosen to go the other route, and use it as a way to visualize data, and those types of things. You’re going to find, when you start working for a newspaper, and they see that you can do this, they’ll be inclined to stereotype you as the “data geek” who can’t write. And that’s a frustration that I’ve had in the past, not by my bosses, my bosses have all been totally cool with it, but other people in the newsroom that don’t know you as well. If someone has a simple math problem or something, they’ll want to run it by you, and have you double-check it, because you’re the numbers person. Pushing back against that, to some extent, has been a really important thing in my career, to not fall into something I didn’t want to fall into using these skills, just cause I have these skills. You can take them and do whatever you want with them. You just have to be persistent, and figure out what you want.

As far as how to get started, you’ll have a number of difficulties if you start at a super-small newspaper. You won’t have software, you won’t have time, you won’t have other people around you, unless you pick the right place, who understand what you’re bringing to the table. I think that’s a frustration for a lot of people I know of who started in smaller places. Of course, it depends on the place. There are some places where they want to have those skills, some of them are actually creating CAR desks at the small newspapers, because they really understand the value. But if you end up in a place, whether it’s big or small, that doesn’t understand the value of what you bring, then you’re not going to be able to grow as well. I think there’s a lot of people out there now, at big papers and at organizations like ours, who are looking to fill that niche. Every few weeks, someone calls me asking if I know of anybody who can fill whatever CAR position at whatever newspaper, and I know I’m not the only one who gets those calls. So there’s plenty of openings out there for that, and so I think the trick there is to get to know the people in the community.

There are so few people who know how to do this stuff, and so many openings, and so much demand, that this is really the one position in journalism, at least that I can think of right now, where they can’t fill the jobs fast enough.

Related posts you might enjoy:

• March 28, 2010 -- Data Delver: Mark Schaver, Louisville Courier
• February 13, 2010 -- Data Delver: Mo Tamman, Wall Street Journal
• March 8, 2010 -- Data Delver: Anthony DeBarros, USA Today
• March 8, 2010 -- Data Delver: Jennifer LaFleur, ProPublica
• February 12, 2010 -- Data Delver: William Hartnett, Palm Beach Post
• February 20, 2010 -- Data Delver: MaryJo Webster, Pioneer Press