Pages Web Design

Step 1: Input tags

The Del.icio.us social bookmarking system runs much of my aggregator: users see a post they like and bookmark it in Del.icio.us with a special tag.

The first step in Pipes is to collect the input (right). Pull the "Text Input" module (above) onto your Pipes work space. This lets you collect user input. Give it both a name and a prompt. In most instances it's fine that these be the same as the prompt won't be visible in the end. It's good to put something down in Debug for later on in the Pipes process.

Step 2: Construct the RSS call

We take our two input tags and use them to construct an URL by using the "URLBuilder". The base URL is Del.icio.us's RSS feed (http://del.icio.us/rss/). The URL builder adds the user input then the tag input to give us a valid URL (http://del.icio.us/rss/user/tag/).

Step 4: Grab the feed

Yahoo Pipes' "Fetch" module takes that URL input and turns it into an RSS feed. Shown to the right is Fetch with the final "Pipe Output".

See it in action

You can see how this fits together by going to my Del.icio.us Sample page on Pipes. You can make a copy and play with it yourself. Add "&user=username&tag=tagname&_render=rss" to that URL and you've made it a feed.

Okay so I've turned a Del.icio.us RSS feed into... a Yahoo Pipes RSS feed with identical input and output. Well, we're only getting started. Our input tags can be reused for other searches and spliced together inside of a more elaborate Pipe. That's where the fun starts and I'll get there soon.

The other advantage of sending things through Pipes is that we can easily rename fields. Del.icio.us, Flickr and other services often extend RSS standards by including metadata in "dc" fields, an abbreviation for the Dublin Core standards extension. A recent entry from my Del.icio.us feed includes this:

     <dc:creator>martin_kelley</dc:creator>
     <dc:date>2007-03-15T05:18:53Z</dc:date>
     <dc:subject>tech tech.design</dc:subject>

Standard PHP parsers like MagpieRSS and SimplePie often have trouble pulling dc data. With Pipes you can rename the fields you like; in theory that should make them more accessible to the parsers. You can also combine fields and use Pipes' Regex module to operate on them with regular expressions.

Don't assume that your website is easy to navigate. One of the neatest things about the web is that we have instant feedback on use. With just a little tracking we can see what pages people are looking at, how they're finding our site and what they're doing once they're here.

Javascript Trackers:

My most advanced sites are currently using four different tracking methods. Most utilize javascript "bugs," tiny snippets of code that send individual results to an advanced software tracking system. I put the code inside a Moveable Type "Modules Template" which is automatically imported to all pages. Installing a new system is as easy as cutting-and-pasting the javascript into the Template and rebuilding the site.

AXS Visitors Tracking System
This software installs on your server but don't let that scare you: this is one of the easiest installations I've ever seen. AXS gives you great charts of usage: you can narrow it specific pages on your site, or even particular search engines or search phrases.

There's also a option to view the lastest traffic by visitor. I love watching this! You can see how individuals are using the site and where they're navigating. I've been able to identify different types of visitors this way and understand the complexity of the audience.

It doesn't seem like AXS is not being developed anymore. The latest stable version came out over two years go, which is a shame.

HitTail
This service watches search-engine links and makes recommendations for new keywords. I wrote about this service yesterday in Blogging for the Long Tail.

Reeferss.com
This is a simple simple bit of software. Like every other tracking system it keeps track of referrers: search engines and websites that bring traffic to your site. But unlike the others that's all it does. Why care then? It provides a real-time RSS feed of these visitors. I bring the feed into my "Netvibes" page (a customized start page, see below) and scan the results multiple times a day.

Google Analytics
The internet's gatekeeper bought the Urchin analytics company in April 2005 and relaunched the product as Google Analytics shortly thereafter. This is becoming an essential tracker. It's free and it's powerful, though I haven't been as impressed by it as others have. See its Wiki page for more.

Internet Trackers:

It's easy to find out what people are saying about you online.

Technorati
This service tracks blogs but you don't need to have a blog to use it, for Technorati will tell you where blogs are linking. Give it your URLs (or those of your competitors!) and you'll know whenever a blogger puts in a link to you. You can also give it keywords and find out when a blog uses them.

Google Blog Search
Google can also let you follow blog references or keyword mentions on the blogs. Google will also track beyond blogs of course. Type "site:www.yourdomain.com" into the main Google search page and you'll see who's linking to your site (or to the competition). There are lots of other services that track blogs and mentions--Sphere, Bloglines, etc. They all have different strengths so try them and see what you think.

Feedburner
The best RSS massager has always focused on ways to track your RSS feed. They've recently introduced page tracking software too. It looks great but I just installed it this week. I still have to see if it's as good as Feedburner's other offerings.

Keeping on top of this flow of data:

It's easy to get overwhelmed by all of this information. Most of the tracking services provide RSS feeds (See The Wonders of RSS Feeds for an intro). I use Netvibes, a customized start page, to pull these all together into a single page that I can scan every morning. Here's a screenshot of part of my Netvibes tracking page--the full page currently shows fourteen tracking feeds on one screen:

So why is tracking important to SEO?

With tracking you find out what people are looking for on the internet. This helps you create pages and services that people will want to find. You might be surprised to see what they're already finding on your site. Some examples:

• Analyzing one site, I noticed that few pages I thought were obscure were bringing in high Google traffic. I looked at these pages again and realized they did a good job of describing the company's mission. I consequently redesigned the site homepage to feature them and I made sure that those pages contained direct links to its most important services.

• When I started work for another client I looked at their site and suspected that they're most important articles were not being seen--visitors had to click through about four times to get to them. Six months of tracking confirmed my hunch and gave me the hard data to convince the executive director that we made some small modifications to the design. Having this strong content linked right off the homepage helped bring in Google traffic.

About eighteen months ago I started a "links blog" of interesting Quaker links, incorporated as a sidebar on my popular "QuakerRanter" personal blog. I eventually gave the links their own URL (QuakerQuaker.org) and invited others to join the linking. I always stumble when trying to tell people what QuakerQuaker is all about. The best definition is that its a "collaboratively edited blog aggregator" but that's a horribly tech description.

The rise of blogs is creating the necessity for these sort of theme-based aggregators. This morning I stumbled on Original Signal, a new site that organzes the best Web 2.0 blogs. A site called PopURLs does the same for "the latest web buzz." A site called SolutionWatch has written about these in Tracking the web with Single Page Aggregators. We're all on to something here. I suspect that sometime this fall some clever person will coin a new term for these sites.

An important part of the redesign was an automatic keyword generator. Posts were run through a script that automatically pulled out keywords from the text. My 2003 article, Going all the way with Movable Type generated the following tags, which appear as links after the post:

• choate
• nonviolence
• curly quotes
• personal blog
• php
• org
• moveabletype
• movable type
• hmtl
• dashes
• textile
• garbage
• blogs

Following the links takes you to similarly-tagged articles. At least that's the conceit. When you follow a tag's link you're simply doing a site search for that keyword. A little htaccess rewrite magic is making the result look like it's a static category page.

"Fine and well" you're thinking, "big deal." Well, here's what's cool. There are 225 entries on the QuakerRanter blog. Google's just gone through and indexed the site and is now claiming it contains 1300 pages. Each tag is being indexed as its own page. Every time I mention any interesting term, it becomes a page that Google indexes and delivers to its searchers.

Which brings us to today's cool piece from the access logs. In December of 2004 a rather innocent post on Quaker Ranter became the center of a mini-whirlwind on the political blogs when it mentioned that I had gotten a call from a CBS News publicist interested in Nonviolence.org. All political blogs get publicity calls from news and opinion think tanks trying to suggest (or plant) stories but no one's supposed to talk about it. I only mentioned it because it was so unusual. One of the blogs denouncing the liberal conspiracy my post revealed was the somewhat slimy Little Green Footballs. After a few weeks the denunciations died down.

But this morning, someone looked up littlegreenfootballs in Google and came to my site. Because of my automatic keyword generator, tags, and static-loooking links, I'm now the number two entry, on two three-year old posts, now relocated to a days old quakerranter.org. Cool.

This mixing and matching of content and rich manipulation of data is sometimes lumped together in the cool bu zzphrase folksonomy. Note that none of what I've done is a tricking of Google. Every tag is really going to a page with that content. These are "natural" and "organic" search results in the lingo of SEO. I'm just presenting my information in multiple formats that appeal that the widest array of audiences.

For what it's worth, I don't think I deserve #2 status for "littlegreenfootballs" and I don't think Google will keep it there for long. It's a bit odd that they have elevated that particular term so high and no others tags seem so stratospheric.

Positive Results:

As of February 2007, Google indexes 3,540 pages on QuakerRanter.org, a blog of only 239 posts. In December 2006 30% of my Google visits were to one of the "tags" page. Reconfiguring the blog in this kind of tag-intensive way has more than doubled search engines visits, again in a very natural and organic way. Adding tags has simply made what I've written more accessible to search engines. Very cool.

Negative Ramifications:

Shortly after installing this new system, my servers started periodically crashing (about once/week). The problem would be multiple MT-Search processes overloading the memory.

My guess is that a search engine spider came along and started indexing all of the tags. Each link initiated a search query in Movable Type. The built-in search for Movable Type is just not able to handle this volume of traffic.

I installed Fast Search to solve the problem (tip of the hat to Al-Muhajabah). It took awhile: Fast Search required a MySQL upgrade at my host. After that I needed to install these plugin fixes. Then it was fine-tuning the htaccess files. It was been more work than I initially expected and the tag results now forward to a funny URL that Google doesn't love as much.

Even websites without PhD designers need to marry a simple outward appearance with a more complicated set of calculations around intended audiences. The average visitor looks at one or two pages on a site and then hits the back button. Often they'll be following a search link and looking at a page buried deep in your site. They'll be there seeking out specific information and you only have about twenty seconds to pitch your site and keep them there. You need to give them a very concise description of yourself or product and you need to entice them with related material.

Any site that consists of more than three pages presents visitors with more information than they can handle. Good design works to funnel visitors to the specific content they are looking for. It's relatively easy to get a first-time visitor but successful websites keep them on your site and give them reasons to return. The key to this is defining your audience and presenting your material with them in mind.

Once you've identified your constituency and built your design, the next step is release. You don't want to pander to a potential audience, but instead converse with them. It's fine to mix different elements of your life together and to write creatively off-topic once in awhile. There are a thousand generic websites crammed full of useless bu zzphrases and unused featured. What you want is one that will have a voice, that builds a niche that no one else might ever have identified. When it comes time to produce content, forget all the slick marketing calculations you've done and let your quirkiness shine.