Photograph Web Design

Step 1: Pick a community designator

I've been using the community name followed by a dot. The prefix goes in front of category description to make a set of unique tags for the aggregator. When someone wants to add something for the site they tag it with this "community.category" tag. In my example, when someone wants to list a new Quaker blog they use "quaker.blog", "quaker" being the community name, "blog" being the category name for the "New Blogs" page.

Step 2: Collect the community prefix and category name in Pipes

You begin by going into Pipes and pulling over two text inputs: one for the community prefix, the other for the specific category.

Step 3: Construct these into tags

Now use the "String Concatenation" module to turn this into the "community.category" model. The community input goes into the top slot, a dot is the second slot and the category input goes into the last slot.

Now, when you have a tag in Flickr with a dot in it, Flickr automatically removes it in the resultant RSS feed. So with Flickr you want your tag to be "communitycategory" without a dot. Simple enough: just pull another "String Concatenation" module onto your Pipes work space. It should look the same except that it won't have the middle slot with the dot.

Step 4: Turn these tags into RSS URLs

Pull three "URLBuilder" modules into Pipes, one for each of the services we're going to query. For the Base, use the non-tag specific part of the URL that each service uses for its RSS feeds. Here they are:

Under path elements, put the correct tag: for Del.icio.us and Google it should be the community.category tag, for Flickr the dot-less communitycategory tag.

Step 5: Fetch and Dedupe

Fetch is the Pipes module that pulls in URLs and outputs RSS feeds. It can also combine them. Send each URLBuilder output into the same Fetch routine.

Since it's possible that you'll might have duplicate posts, use the "Unique" module to deduplicate entries by URL. Through a little trial and error I've determined that in cases of duplicates, feeds lower in the Fetch list trump those higher. In the actual Pipe powering my aggregator I pull a second Del.icio.us feed: my own. I have that as the last entry in the Fetch list so that I can personally override every other input.

Step 6: Sort by Date

With experimentation it seems like Pipes orders the output entries by descending date, which is probably what you want. But I want to show how Pipes can work with "dc" data, the "Dublin Core" model that allows you to extend standard RSS feeds (see yesterday's post for more on this).

Google Blog Search and Del.icio.us feeds use the "dc:date" field to record the time when the post was made. Flickr uses "dc:date.Taken" to pass on the photograph's metadata about when it was taken. Pipes' "Rename" module lets you copy both fields into one you create (I've simply used "date"), which you can then run through its "Sort" module. Again, it's a moot point since Pipes seems to do this automatically. But it's good to know how to manipulate and rename "dc" data if only because many PHP parsers have trouble laying it out on a webpage.

Update: it's all moot: according to a ZDNet blog, "Pipes now automatically appends a pubDate tag to any RSS feed that has any of the other allowable date tags." This is nice: no need to hack the date every time you want to make a Pipe!

Step 7: Output

The final step for any Pipe is the "Pipe Output" module.

In action

You can see this published Pipe here, and copy and play with it yourself. The result lets you build an RSS feed based on the two inputs.

Syndication feeds are the lingua franca powering all the cool new websites. It doesn't matter what blogging platform you use or what operating system you're on: if your software provides an RSS feed I can mix and match it and use it to pull in content to my site.

Examples 1: Photographs: I email all of my adorable kid pictures to the photo sharing site Flickr, which then provides a syndication feed ("here"). I use a little fancy patch of coding on my website to pull in the information about the latest photos (location, caption, etc) so that I can display them on my homepage. Whenever you go to my Theo age you'll see the latest Flickr photos of him.

Example 2: Bookmarks. I also use the "social bookmarking" system with the odd name of del.icio.us. When I find a page I want to bookmark, I click a Delicious button in my browser, which opens a pop-up window. I write a description, pick a category or two and hit save. Deliciouis then provides an RSS syndication feed which I can use to pull together a list of my latest bookmarks and display it on my website. Wave a few magic wands of complication (pay no attention to the man behind the curtain!) and you have the main trick behind Quakerquaker.org.

I've simplified both examples a bit but you probably get the point. Syndication feeds are the secret behind blog readers like Bloglines and email subscription services like the one's I provide for quakerquaker.org.

New to me is the concepts around the Well-Formed Web. As described by Kevin Donahue "The layman's premise of the Well-Formed Web is that each site will have drill-down feeds - a top level feed, item specific feeds, and so on." What this means is that you don't just have one single RSS feed on a site (your latest ten posts) but RSS feeds on everything. Every category get its own unique feeds (e.g., the last ten posts about web design) and every post gets its own unique feed tracking its comments (e.g., this feed of comments from my "Introducing MartinKelley.com" post). It certainly seems a bit like overkill but computers are doing all the work and the result gives us a multi-dimensionality that we can use to pull all sorts of neat things together.