Xml Web Design

Step 1: Input tags

The Del.icio.us social bookmarking system runs much of my aggregator: users see a post they like and bookmark it in Del.icio.us with a special tag.

The first step in Pipes is to collect the input (right). Pull the "Text Input" module (above) onto your Pipes work space. This lets you collect user input. Give it both a name and a prompt. In most instances it's fine that these be the same as the prompt won't be visible in the end. It's good to put something down in Debug for later on in the Pipes process.

Step 2: Construct the RSS call

We take our two input tags and use them to construct an URL by using the "URLBuilder". The base URL is Del.icio.us's RSS feed (http://del.icio.us/rss/). The URL builder adds the user input then the tag input to give us a valid URL (http://del.icio.us/rss/user/tag/).

Step 4: Grab the feed

Yahoo Pipes' "Fetch" module takes that URL input and turns it into an RSS feed. Shown to the right is Fetch with the final "Pipe Output".

See it in action

You can see how this fits together by going to my Del.icio.us Sample page on Pipes. You can make a copy and play with it yourself. Add "&user=username&tag=tagname&_render=rss" to that URL and you've made it a feed.

Okay so I've turned a Del.icio.us RSS feed into... a Yahoo Pipes RSS feed with identical input and output. Well, we're only getting started. Our input tags can be reused for other searches and spliced together inside of a more elaborate Pipe. That's where the fun starts and I'll get there soon.

The other advantage of sending things through Pipes is that we can easily rename fields. Del.icio.us, Flickr and other services often extend RSS standards by including metadata in "dc" fields, an abbreviation for the Dublin Core standards extension. A recent entry from my Del.icio.us feed includes this:

     <dc:creator>martin_kelley</dc:creator>
     <dc:date>2007-03-15T05:18:53Z</dc:date>
     <dc:subject>tech tech.design</dc:subject>

Standard PHP parsers like MagpieRSS and SimplePie often have trouble pulling dc data. With Pipes you can rename the fields you like; in theory that should make them more accessible to the parsers. You can also combine fields and use Pipes' Regex module to operate on them with regular expressions.

Syndication feeds are the lingua franca powering all the cool new websites. It doesn't matter what blogging platform you use or what operating system you're on: if your software provides an RSS feed I can mix and match it and use it to pull in content to my site.

Examples 1: Photographs: I email all of my adorable kid pictures to the photo sharing site Flickr, which then provides a syndication feed ("here"). I use a little fancy patch of coding on my website to pull in the information about the latest photos (location, caption, etc) so that I can display them on my homepage. Whenever you go to my Theo age you'll see the latest Flickr photos of him.

Example 2: Bookmarks. I also use the "social bookmarking" system with the odd name of del.icio.us. When I find a page I want to bookmark, I click a Delicious button in my browser, which opens a pop-up window. I write a description, pick a category or two and hit save. Deliciouis then provides an RSS syndication feed which I can use to pull together a list of my latest bookmarks and display it on my website. Wave a few magic wands of complication (pay no attention to the man behind the curtain!) and you have the main trick behind Quakerquaker.org.

I've simplified both examples a bit but you probably get the point. Syndication feeds are the secret behind blog readers like Bloglines and email subscription services like the one's I provide for quakerquaker.org.

New to me is the concepts around the Well-Formed Web. As described by Kevin Donahue "The layman's premise of the Well-Formed Web is that each site will have drill-down feeds - a top level feed, item specific feeds, and so on." What this means is that you don't just have one single RSS feed on a site (your latest ten posts) but RSS feeds on everything. Every category get its own unique feeds (e.g., the last ten posts about web design) and every post gets its own unique feed tracking its comments (e.g., this feed of comments from my "Introducing MartinKelley.com" post). It certainly seems a bit like overkill but computers are doing all the work and the result gives us a multi-dimensionality that we can use to pull all sorts of neat things together.