Taming RSS

2004 is obvi­ously the year of RSS, with arti­cle pop­ping up left and right in main­stream pub­li­ca­tions. How­ever, RSS can also be a source of much stress, if you sub­scribe to too much.

A few weeks ago, my list of sub­scribed feeds went over 300. That was the begin­ning of a sober­ing exper­i­ment. While it is tech­ni­cally pos­si­ble to fol­low 300 sites via RSS, it’s not for the faint-hearted. I’ve since been prun­ing the list a lit­tle as it became more and more time con­sum­ing to go through all the entries. While I felt like I must be fail­ing some­how, Sebastien Paquet pointed out that the median num­ber of sub­scrip­tions peo­ple have is under 100.

I sus­pect this is where the power laws actu­ally become use­ful. Because some blogs are dis­pro­por­tion­ally read, they can be seen as flag-bearers in the blo­gos­phere. Because they are so pow­er­ful, they can eas­ily shape opin­ions in the blog world. And because they do so, one can limit the num­ber of blogs they read in order to get an idea as to con­sen­sus among blog­sters. This is great in that those pow­er­ful blog­gers become edi­tors of sorts.

There is, how­ever, a prob­lem with that. As recently reported in a Wired News story, the most-read weblog­gers aren’t nec­es­sar­ily the ones with the most orig­i­nal ideas. This means that the power laws suc­ceed in a main­stream­ing of ideas but fails in terms of com­ing up with new ones. This, unfor­tu­nately, means that there is a bit of a pack men­tal­ity among power blog­gers which can only be counter-balanced by read­ing blogs that are not as popular.

But blogs rep­re­sent only part of the RSS world. If one adds news sources from main­stream pub­li­ca­tions, one gets a fuller pic­ture of a sub­ject, mix­ing expert opin­ion (from the blog­gers) which gen­eral overviews (from the media). From this mix, one can get a fuller pic­ture. What we now need is a tool that would cre­ate some­thing akin to a self-organizing sys­tem within the RSS world. Tools like Blogdex pro­vide an idea as to what’s pop­u­lar right now; Tools like Feed­ster give search capa­bil­i­ties; and tools like Share your OPML pro­vide data as to what peo­ple sub­scribe to.

The next step is find­ing a merger of those three cat­e­gories, along with some bet­ter track­ing mech­a­nism as to what is actu­ally read and what links are fol­lowed vs. what is sub­scribed to. If, based on the stuff I read (and not nec­es­sar­ily the stuff I sub­scribe to), I could get some changes in behav­ior in my aggre­ga­tor (as in “the fol­low­ing arti­cles in feeds you sub­scribe to are seen as impor­tant by other peo­ple who read sim­i­lar stuff and are related to cat­e­gories you are inter­ested in”), I might be able to tame the flow of infor­ma­tion I get. Think of it as the equiv­a­lent of the karma sys­tem on Slash­dot. This would give me an idea as to what is pop­u­lar and of inter­est to me.

The next step would be to also pro­vide a serendip­ity fac­tor. Ideas that are out on their own should have their own bas­ket. If a par­tic­u­lar site is a good source of orig­i­nal con­tent, then that source should move up in my per­sonal rank­ing if I am sub­scribed to it.

Of course, the clas­si­fi­ca­tion pro­vided in RSS 2.0 and in OPML also needs to be con­sid­ered as part of this. If there was a way to sort feeds by cat­e­gories (and iden­tify cat­e­gories based on how peo­ple clas­si­fied things in their OPML file), it would make things eas­ier. For exam­ple, I clas­sify some feeds in the Gear cat­e­gory but oth­ers might clas­sify them in the Gad­gets cat­e­gory. I should be able to then cre­ate an asso­ciate between those two words so that when I peruse someone’s else feed, entries and blogs listed under Gad­gets would pop up in my Gear section.

This seems like a lot to code but could truly give some seman­tic to the web. Already, the world wide web, as it was used pre-RSS is becom­ing an archival medium and RSS is becom­ing the updated world (as a side note, this is going to have a huge impact on web-side design and mar­ket­ing as one has to rethink how to reach reader in a space where all entries look alike).