Journalism 2.0

Editor’s Note: Today’s guest writer is Rick Martin, a Tokyo-based freelance writer. Read more from Rick at www.1rick.com/blog and follow him on Twitter at @1rick.

I’m not a programmer. But these days I’m starting to see how some programming skills can really make a big difference to my productivity as a writer. As I pretty much live inside my RSS readers, I find myself bouncing around between different websites copying and pasting feed links far more than I should. For example, if I want to create RSS feeds for the keyword ‘obama’, I don’t want to have to go to Google News, Yahoo News, Delicious, Flickr, Bing, Youtube and all those other services to retrieve those feeds. Try as I might, I couldn’t find any web service that would produce feeds for a given search term across multiple social media services and news sites.

This was a problem.

Solution: I decided to try to program such a tool on my own. Again I’m not a programmer, but I started with ‘Hello World’ and just researched other snippets of PHP code that I thought would do the job. I’d like to walk you through the process because if you’re new to programming this is a good way to get your feet wet.

In a nutshell, my tool would take the link structures for RSS feeds like this:

http://www.youtube.com/rss/tag/KEYWORD.rss

and replace the ‘keyword’ with a term that you would like to monitor using a simple string replacement command. So if you input ‘obama’, you get:

http://www.youtube.com/rss/tag/obama.rss

First we’ll need to whip up an index.html file to hold our input form, and a feeds.php file to display the feed results. To make these you can use your favorite text editor. Today I’m using Smultron. So lets get started!

Step 1. Create index.html and add basic html tags.

Step 2: Add a search form to index.html

This form will prompt the user to enter the keyword that they want to generate a feed for. So for example, if I want to monitor news about ‘obama’, I will enter ‘obama’ into the text input box. You can also add some explanatory text for the user just before the form if you wish. But I’ll leave that up to you. If you’d like to get an idea how this form will look in a webpage, skip ahead to the final step.

Step 3: Create the ‘feed results’ page

You might have noticed the ‘feeds.php’ reference in the form above. We’ll make that page now. So create ‘feeds.php’.

After you’ve done that, you’ll need to declare the RSS feed link structures for your favorite media services. I’m going to use Delicious, Google News, Google Blogs, Yahoo News, Bing, Twitter, Flickr, and YouTube. For each RSS link, I’ll assign a variable. So for the YouTube feed that I mentioned above, I’ll call that ‘$youtube’. So now the ‘$youtube’ variable will be representing the string of text that makes up the RSS link http://www.youtube.com/rss/tag/KEYWORD.rss.

Notice that I’m using REPLACEME as the keyword in all of these links, because I’m planning to substitute in the string of text that the user inputs in the text box. Simple, right?

Step 4: Replace ‘REPLACEME’ with ‘SEARCHTERM’

Now we’re going to do a simple string replacement command to replace all instances of ‘REPLACEME’ with the user-inputted text. This will generate new RSS feed links that we can then use however we choose. Note that I’m assigning a new variable to the resulted links. So our old link template, $youtube, now becomes $youtubenew, and so on down the line for all the media services.

Step 5: Get ready to display feed results

So we’ve generated the feeds we wanted. But we still haven’t displayed them yet. So let’s do that now. Add html, head, and body tags. You can add some meta info and CSS if you like as well. We’re almost done!

Step 6: Display feed results in unordered list

A simple unordered list will do to display the new generated feeds. We’re going to do a simple ‘echo’ command to print out the new feeds here. Be sure to close out your body and html tags at the end of the document.

Step 7: Test it out!

I have a working version of this tool on my website. But you can easily upload the index.html and feeds.php files to a subdirectory on your own website and use it from there.

I really prefer to use it embedded in a Netvibes page via the webpage widget (see below), because then I can generate feeds for a desired term and copy and paste the resultant feeds directly into a new Netvibes page. It’s very handy when creating media monitoring ‘dashboards’.

Taking it one step further

Ideally I’d like this tool to parse the ten most recent items for each feed generated. SimplePie might be a good solution for this, and to that end I’ve been trying to absorb this awesome tutorial from wunderkind Tom Rogers. Unfortunately the feeds aren’t quite playing nice yet, but I’ll keep working on it.

I’ve also been experimenting creating feeds for sites that don’t have any feeds. For this I’ve been generating RSS feeds using Dapper.net, an absolutely wonderful data scraping service. This string replacement function doesn’t always work with their links though, so your milage may vary.

Some final thoughts

I’m sure that all my code above wasn’t the most efficient or elegant, but the end result does make research a lot easier for me. If there are more seasoned programmers out there, I invite your feedback (please be gentle). For the noobs out there, I really encourage you to try adding some programming skills to your repertoire. I guarantee that you’ll start thinking about problems differently once you have a new set of tools in your box.

And if that doesn’t convince you that programming is a skill every journalist should have, check out my friend Chris who programmed his awesome Tools For News site from scratch and landed a job as a result. It’s a damn inspiring story and I’d likely not have given programming a second look if it weren’t for Chris’s fun projects.

Admittedly, programming is not the easiest thing to learn from scratch. But for those who want to give it a try, I have assembled some beginning resources on my blog that will help you learn programming from scratch. And if you have any resources that have worked well for you, I’d love to hear them.

I’m hoping to launch a local Tokyo site over the next few weeks, and this feed generator tool will certainly help out contributors who need to monitor local news their corner of Tokyo. Similarly, I hope it helps some of you readers out there as well.

NOTE: Here are the source files if you want to play around with them or install them on your own site: http://1rick.com/feedtoolsrc