Rss Web Design

Step 1: Pick a community designator

I've been using the community name followed by a dot. The prefix goes in front of category description to make a set of unique tags for the aggregator. When someone wants to add something for the site they tag it with this "community.category" tag. In my example, when someone wants to list a new Quaker blog they use "quaker.blog", "quaker" being the community name, "blog" being the category name for the "New Blogs" page.

Step 2: Collect the community prefix and category name in Pipes

You begin by going into Pipes and pulling over two text inputs: one for the community prefix, the other for the specific category.

Step 3: Construct these into tags

Now use the "String Concatenation" module to turn this into the "community.category" model. The community input goes into the top slot, a dot is the second slot and the category input goes into the last slot.

Now, when you have a tag in Flickr with a dot in it, Flickr automatically removes it in the resultant RSS feed. So with Flickr you want your tag to be "communitycategory" without a dot. Simple enough: just pull another "String Concatenation" module onto your Pipes work space. It should look the same except that it won't have the middle slot with the dot.

Step 4: Turn these tags into RSS URLs

Pull three "URLBuilder" modules into Pipes, one for each of the services we're going to query. For the Base, use the non-tag specific part of the URL that each service uses for its RSS feeds. Here they are:

Under path elements, put the correct tag: for Del.icio.us and Google it should be the community.category tag, for Flickr the dot-less communitycategory tag.

Step 5: Fetch and Dedupe

Fetch is the Pipes module that pulls in URLs and outputs RSS feeds. It can also combine them. Send each URLBuilder output into the same Fetch routine.

Since it's possible that you'll might have duplicate posts, use the "Unique" module to deduplicate entries by URL. Through a little trial and error I've determined that in cases of duplicates, feeds lower in the Fetch list trump those higher. In the actual Pipe powering my aggregator I pull a second Del.icio.us feed: my own. I have that as the last entry in the Fetch list so that I can personally override every other input.

Step 6: Sort by Date

With experimentation it seems like Pipes orders the output entries by descending date, which is probably what you want. But I want to show how Pipes can work with "dc" data, the "Dublin Core" model that allows you to extend standard RSS feeds (see yesterday's post for more on this).

Google Blog Search and Del.icio.us feeds use the "dc:date" field to record the time when the post was made. Flickr uses "dc:date.Taken" to pass on the photograph's metadata about when it was taken. Pipes' "Rename" module lets you copy both fields into one you create (I've simply used "date"), which you can then run through its "Sort" module. Again, it's a moot point since Pipes seems to do this automatically. But it's good to know how to manipulate and rename "dc" data if only because many PHP parsers have trouble laying it out on a webpage.

Update: it's all moot: according to a ZDNet blog, "Pipes now automatically appends a pubDate tag to any RSS feed that has any of the other allowable date tags." This is nice: no need to hack the date every time you want to make a Pipe!

Step 7: Output

The final step for any Pipe is the "Pipe Output" module.

In action

You can see this published Pipe here, and copy and play with it yourself. The result lets you build an RSS feed based on the two inputs.

Step 1: Input tags

The Del.icio.us social bookmarking system runs much of my aggregator: users see a post they like and bookmark it in Del.icio.us with a special tag.

The first step in Pipes is to collect the input (right). Pull the "Text Input" module (above) onto your Pipes work space. This lets you collect user input. Give it both a name and a prompt. In most instances it's fine that these be the same as the prompt won't be visible in the end. It's good to put something down in Debug for later on in the Pipes process.

Step 2: Construct the RSS call

We take our two input tags and use them to construct an URL by using the "URLBuilder". The base URL is Del.icio.us's RSS feed (http://del.icio.us/rss/). The URL builder adds the user input then the tag input to give us a valid URL (http://del.icio.us/rss/user/tag/).

Step 4: Grab the feed

Yahoo Pipes' "Fetch" module takes that URL input and turns it into an RSS feed. Shown to the right is Fetch with the final "Pipe Output".

See it in action

You can see how this fits together by going to my Del.icio.us Sample page on Pipes. You can make a copy and play with it yourself. Add "&user=username&tag=tagname&_render=rss" to that URL and you've made it a feed.

Okay so I've turned a Del.icio.us RSS feed into... a Yahoo Pipes RSS feed with identical input and output. Well, we're only getting started. Our input tags can be reused for other searches and spliced together inside of a more elaborate Pipe. That's where the fun starts and I'll get there soon.

The other advantage of sending things through Pipes is that we can easily rename fields. Del.icio.us, Flickr and other services often extend RSS standards by including metadata in "dc" fields, an abbreviation for the Dublin Core standards extension. A recent entry from my Del.icio.us feed includes this:

     <dc:creator>martin_kelley</dc:creator>
     <dc:date>2007-03-15T05:18:53Z</dc:date>
     <dc:subject>tech tech.design</dc:subject>

Standard PHP parsers like MagpieRSS and SimplePie often have trouble pulling dc data. With Pipes you can rename the fields you like; in theory that should make them more accessible to the parsers. You can also combine fields and use Pipes' Regex module to operate on them with regular expressions.

Just about every native Web 2.0 applications comes built-in with multiple RSS feeds. But in the real world, websites are built using an almost-infinite number of content management systems and web development software programs. Sometimes a single website will use different programs for putting its contents online and sometimes a single organization spreads its functions over multiple domains.

Step 1: Make it Del.icio.us

To begin, sign up with Del.icio.us, the popular "social bookmarking" web service (similar services can be easily adapted to work). Then add a "post to Del.icio.us" button to your browser's toolbar following the instructions here. Now whenever you put new content up on your site, go that new page, click on your "post to Del.icio.us" button and fill out a good title and description. Choose a tag to use. A tag is simply a category and you can make it whatever you want but "mysites" or your business name will be the easiest to remember. Hit save and you've started an RSS feed.

How? Well, Del.icio.us turns each tag into a RSS feed. You can see it in all its machine code glory at del.icio.us/rss/username/mysites (replacing "username" with your username and "mysites" with whatever tag you chose).

Now you could just advertise that Del.icio.us RSS feed to your audience but there are a few problems doing this. One is that Del.icio.us accounts are usually personal. If your webmaster leaves, then your published RSS feed will need to change. Not a good scenario, especially since you won't even be able to tell who's still using that old feed. Before you advertise your feed you should "future proof" it by running it through Feedburner.

Cloak that Feed

Go to Feedburner.com. Right there on the homepage they invite you to type in a URL. Enter your Del.icio.us feed's address and sign up for a Feedburner account. In the field next to feed address give it some sensible name relating to your company or site, let's say "mycompany" for our example. You'll now have a new RSS feed at feeds.feedburner.com/mycompany. Now you're in business: this is the feed you advertise to the world. If you ever need to change the source RSS feed you can do that from within Feedburner and no one need know.

The default title of your Feedburner feed will still show it's Del.icio.us roots (and the webmaster's username). To clear that out, go into Feedburner's "Optimize" tab and turn on the "Title/Description Burner," filling it out with a title and description that better matches your feed's purpose. For an example of all this in action, the Del.icio.us feed that powers my tech link blog and its Feedburner "cloak" can be found here:

• http://del.icio.us/rss/martin_kelley/tech
• http://feeds.feedburner.com/techlinksblog

Get that Feed out there

Under Feedburner's "Publicize" tag there are lots of neat features to republish your feed yourself. First off is the "Chicklet chooser" which will give you that ubiquitous RSS feed icon to let visitors know you've entered the 21st Century. Their "Buzz Boost" feature lets you create a snippet of code for your homepage that will list the latest additions. "Email subscriptions" lets your audience sign up for automatic emails whenever you add something to your site.

Final Thoughts

RSS feeds are great ways of communicating exciting news to your audiences. If you're lucky, important bloggers in your audience will subscribe to your feed and spread your news to their networks. Creating a feed through a bookmarking service allows you to add any page on any site regardless of its underlying structure.

Don't assume that your website is easy to navigate. One of the neatest things about the web is that we have instant feedback on use. With just a little tracking we can see what pages people are looking at, how they're finding our site and what they're doing once they're here.

Javascript Trackers:

My most advanced sites are currently using four different tracking methods. Most utilize javascript "bugs," tiny snippets of code that send individual results to an advanced software tracking system. I put the code inside a Moveable Type "Modules Template" which is automatically imported to all pages. Installing a new system is as easy as cutting-and-pasting the javascript into the Template and rebuilding the site.

AXS Visitors Tracking System
This software installs on your server but don't let that scare you: this is one of the easiest installations I've ever seen. AXS gives you great charts of usage: you can narrow it specific pages on your site, or even particular search engines or search phrases.

There's also a option to view the lastest traffic by visitor. I love watching this! You can see how individuals are using the site and where they're navigating. I've been able to identify different types of visitors this way and understand the complexity of the audience.

It doesn't seem like AXS is not being developed anymore. The latest stable version came out over two years go, which is a shame.

HitTail
This service watches search-engine links and makes recommendations for new keywords. I wrote about this service yesterday in Blogging for the Long Tail.

Reeferss.com
This is a simple simple bit of software. Like every other tracking system it keeps track of referrers: search engines and websites that bring traffic to your site. But unlike the others that's all it does. Why care then? It provides a real-time RSS feed of these visitors. I bring the feed into my "Netvibes" page (a customized start page, see below) and scan the results multiple times a day.

Google Analytics
The internet's gatekeeper bought the Urchin analytics company in April 2005 and relaunched the product as Google Analytics shortly thereafter. This is becoming an essential tracker. It's free and it's powerful, though I haven't been as impressed by it as others have. See its Wiki page for more.

Internet Trackers:

It's easy to find out what people are saying about you online.

Technorati
This service tracks blogs but you don't need to have a blog to use it, for Technorati will tell you where blogs are linking. Give it your URLs (or those of your competitors!) and you'll know whenever a blogger puts in a link to you. You can also give it keywords and find out when a blog uses them.

Google Blog Search
Google can also let you follow blog references or keyword mentions on the blogs. Google will also track beyond blogs of course. Type "site:www.yourdomain.com" into the main Google search page and you'll see who's linking to your site (or to the competition). There are lots of other services that track blogs and mentions--Sphere, Bloglines, etc. They all have different strengths so try them and see what you think.

Feedburner
The best RSS massager has always focused on ways to track your RSS feed. They've recently introduced page tracking software too. It looks great but I just installed it this week. I still have to see if it's as good as Feedburner's other offerings.

Keeping on top of this flow of data:

It's easy to get overwhelmed by all of this information. Most of the tracking services provide RSS feeds (See The Wonders of RSS Feeds for an intro). I use Netvibes, a customized start page, to pull these all together into a single page that I can scan every morning. Here's a screenshot of part of my Netvibes tracking page--the full page currently shows fourteen tracking feeds on one screen:

So why is tracking important to SEO?

With tracking you find out what people are looking for on the internet. This helps you create pages and services that people will want to find. You might be surprised to see what they're already finding on your site. Some examples:

• Analyzing one site, I noticed that few pages I thought were obscure were bringing in high Google traffic. I looked at these pages again and realized they did a good job of describing the company's mission. I consequently redesigned the site homepage to feature them and I made sure that those pages contained direct links to its most important services.

• When I started work for another client I looked at their site and suspected that they're most important articles were not being seen--visitors had to click through about four times to get to them. Six months of tracking confirmed my hunch and gave me the hard data to convince the executive director that we made some small modifications to the design. Having this strong content linked right off the homepage helped bring in Google traffic.

Syndication feeds are the lingua franca powering all the cool new websites. It doesn't matter what blogging platform you use or what operating system you're on: if your software provides an RSS feed I can mix and match it and use it to pull in content to my site.

Examples 1: Photographs: I email all of my adorable kid pictures to the photo sharing site Flickr, which then provides a syndication feed ("here"). I use a little fancy patch of coding on my website to pull in the information about the latest photos (location, caption, etc) so that I can display them on my homepage. Whenever you go to my Theo age you'll see the latest Flickr photos of him.

Example 2: Bookmarks. I also use the "social bookmarking" system with the odd name of del.icio.us. When I find a page I want to bookmark, I click a Delicious button in my browser, which opens a pop-up window. I write a description, pick a category or two and hit save. Deliciouis then provides an RSS syndication feed which I can use to pull together a list of my latest bookmarks and display it on my website. Wave a few magic wands of complication (pay no attention to the man behind the curtain!) and you have the main trick behind Quakerquaker.org.

I've simplified both examples a bit but you probably get the point. Syndication feeds are the secret behind blog readers like Bloglines and email subscription services like the one's I provide for quakerquaker.org.

New to me is the concepts around the Well-Formed Web. As described by Kevin Donahue "The layman's premise of the Well-Formed Web is that each site will have drill-down feeds - a top level feed, item specific feeds, and so on." What this means is that you don't just have one single RSS feed on a site (your latest ten posts) but RSS feeds on everything. Every category get its own unique feeds (e.g., the last ten posts about web design) and every post gets its own unique feed tracking its comments (e.g., this feed of comments from my "Introducing MartinKelley.com" post). It certainly seems a bit like overkill but computers are doing all the work and the result gives us a multi-dimensionality that we can use to pull all sorts of neat things together.