Looking at the history

In order to look forward, it always pays to look back. The dominant standards for the web today are undeniably HTML (or its variances like XHTML) and HTTP. More recently, XML has emerged and, increasingly, RSS is becoming the dominant type of XML for sharing a variety of data.

How did each of those standards become a standard. It is obvious now (hindsight is always 20/20) that standards bodies have relatively little bearing when it comes to influencing the succes of a format. Take, for example, SGML, which was the dominant standardized format for document formatting. It was quickly superceded by HTML which, at the time, was not considered a standard.

The same is true of RSS and other standards for syndication. Formats like ICE, CDF, and NewsML were touted as the future when they were first introduced. However, they’ve recently been superceded by RSS.

And even within the RSS world, formats like RSS 1.0, which was supposed to be more semantically sound, and ATOM, which was supposed to be more forward thinking that RSS 2.0, have been losing the war to RSS 2.0.

Bootstrapping is a social phenomenon

What Dave Winer understood, when he sheperded RSS 2.0 into becoming the dominant mean of delivering syndicated content is that the life and death of a new format is predicated on its widespread adoption. And, in order to increase adoption, one has to make something generic, easy to understand, and simple.

Many of the people in the early days of the syndication space failed to see it as Dave did. We believed that a semantically sound format was better and we were wrong. Purity, it turns out is not always a good thing, especially if it gets in the way of people implementing something.

The same is true of HTML. I’d venture that, from a development standpoint, the biggest boost to HTML was a single menu feature that appeared in early browsers and remains there to this day: view source. In the early days of the web, countless developers learned how to do cool things with HTML by reading the source of pages designed by other people.

In a recent issue of ambidextrous magazine, Jeffrey Schox talks about the three stages of technological development: appropriation, early innovation, and sustainable innovation. Here’s how he describes the appropriation stage:

an issued pattent allows innovators to construct roadblocks behind them as they travel down a particular technological path… During the appropriation stage, patent roadblocks waste time and money… The countries, needing to catch up with the designs and technologies of other countries, should focus on collecting revenue and knowledge streams to fuel later stages of technology development.

While he focuses on hardware and electronics in a globalized marketplace, the same truth can be applied to standards. With few barriers in adopting a new standard and by fostering a culture of appropriation, one can easily establish a base of people who understand a new format. As more people understand it, they start implementing it and, after eventually getting smarter about it, start building on the efforts of previous creators. Eventually, those masses of tinkerers get to a critical point, pushing the new format into areas that were unexpected. Some companies eventually get smart to it and see growth in that area, which triggers them into experimenting with that new format.

Eventually, due to a general agreement among all developers, the format becomes a de facto standard. It does not have to have the imprimatur of a standard body (except for some very late adopters or pockets where such imprint is considered important) and moves forward.

What is interesting is the next stage, the one where standard bodies see the area as hot and decide that they need to play in that field. A good example of that is the ATOM format, which has been enshrined into an IETF approved format, and to date has failed to stop the RSS 2.0 juggernaut.

So what happened?

The amazing thing is how simple the issue is. The reason RSS 2.0 has been winning is that it has developed a following. With every new developer learning RSS 2.0, the format goes stronger and the same is true of every company implementing it. Because it is simple, it’s easy to pick up, which means that new developers can do interesting things with it relatively quickly, giving them a chance to become active members of the community and therefore become hooked on it.

The other issue is in keeping things relatively open, while still maintaining some level of control over the general direction. A successful future standard has to allow people a chance to contribute but, in the end, it also needs some gatekeepers who decide what goes in and what doesn’t. The same truth can be applied to any sofware development cycle: for example, Linux may be a widespread open source phenomenon but the number of people who decide what goes into the core kernel or doesn’t is still relatively limited. The same is true of any successful open source project: some level of centralized decision making and distribution of the work: anyone can contribute but not every contribution makes it into the final product.

I’m now seeing some of the same history repeat itself in the OPML space. It’s a format that is very simple and Dave is working very hard on getting people left and right to support it. It’s the same scenario he’s used to bootstrap the RSS format and to bootstrap concepts like blogging and podcasting into the mainstream. It’s a formula that works: keep it simple to implement, maintain some level of centralized control over the roadmap and then evangelize it left and right until it can no longer be stopped.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Standards as social contracts. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Some History

Before I go into details about Apple’s new offering, I want to give a little background that will clear up some of my confusions. I’ve been involved in the RSS community since 1999, way back when it was just the domain of geeks.

Back in 2000, I made a few suggestions as to how RSS could be improved. At the same, the main version of RSS was version 0.91 and there was some interest in making a new version that would be called RSS 0.92 (yes, it was the alpha days of RSS). So five years ago, I was pushing for crazy concepts like adding a date to an item or finding ways to attach sound files and video files into RSS feeds. Because of that, some people have asked me to opine on things like podcasting and my general contention is that podcasting is a good thing and that the way support for richer files is implemented in RSS is much sounder than what I had offered in the past.

Subsequent battles created a fork in the RSS movement, with one of the main issues being the use of namespaces in RSS. From there came the great split, with RSS 1.0 breaking rank with previous versions of the format, and RSS 2.0 breaking rank with RSS 1.0. Two formats, which moved in parallel. Dave Winer did a great job promoting the 2.0 format and eventually, a majority started supporting it. Since then, a third syndication format (known as ATOM) has popped up and its making its way toward a 1.0 release. With all this, we’re seeing a lot of smart people basically trying to solve some of the same problems, without really working together.

A proposal

Looking at this, I pity the fact that it took us so long to get as far as we’ve gotten. However, with large players now dancing in the syndication space, I am starting to worry that things are going to get worse before they get better. As a result, I’d like to offer a modest proposal: let’s merge all this work and come up with established data sets that are compatible. The use of namespaces for each vendor use is a great idea but shouldn’t one first think about what they are trying to accomplish and look at prior art before trying to reinvent the wheel? Let’s look at the example of today’s announcement from Apple.

RSS Does Media

So podcasting is becoming much bigger. And videocasting is coming soon. How about looking at media use in RSS. Wait, what do you know: Yahoo! has done some of the work already with the media RSS specification (I know this is the second time in as many week that I’ve pulled out the Yahoo! name but it’s because they’ve been doing good work). The specification provides a number of interesting things so I would suggest that Yahoo! and Apple developers sit down together and come up with an agreed upon set of definitions. Here are a few things that I would put on the table for discussion by both entities:

1. A common namespace: it would be nice if they both agreed to a common namespace. I’d reccommend something that does not include a version number (a mistake made in the Apple spec) but it might be nice to have it set as a DTD, which could ease validation.
2. Add media:group to the final specification, it looks like a very valuable one, especially for content that is encoded in more than one way (this will probably be something Apple does not want)
3. Retain media:category and have it replace itunes:category. Here, the Yahoo version seems to provide for more flexibility
4. Replace itunes:explicit and media:adult with media:explicit. What is defined as an adult varies from country to country whereas explicit is well, more explicit.
5. media:text should replace the itunes:subtitle and itunes:summary but it should also get something added to differentiate the two (maybe a content attribute?)
6. itunes:author could be taken care of with media:credit. Maybe this one could be required. The role of owner should be added to it and an extra attribute could be added for email which would cover the whole itunes:owner section
7. itunes:images and media:thumbnail could be merged
8. itunes:block is a good idea and could be created as a new media:block element which would also have a distributor attribute. This distributor attribute would allow to block different distributors moving forward so a creator could decide to distribute certain content only to certain channels.

If both Yahoo! and Apple were to agree to do this, they would end up with a much stronger joint specification and I believe it would also represent a show of good faith from both companies and an understanding that cooperation is good for everyone. I may dream but I hope that we will see this kind of partnership happen, which is why I’d like to ask everyone to make sure to tell their friends about this entry. Together, maybe we can get Apple and Yahoo! to work together on cleaning this stuff up (and anyone else who wants to play in that space, including Microsoft and Google). Otherwise, we will see increasing fragmentation of the markets, which will result in less content for each of the specification proponents.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title RSS and Media: Can’t we all just get along?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

How we got there?

Microsoft is not really a new player in the syndication space. With the release of Internet Explorer 4.0, in 1997, the Redmond giant introduced CDF, a format to push content and software to the operating system. With the craze around push deflating, CDF was pushed in the background.

While such efforts were not that successful, RSS moved stealthily towards the mainstream. As a plucky little standard, it is starting to dominate how a lot of notification is being done. So Microsoft decided, wisely, to join the RSS bandwagon.

However, old habits die hard and just embracing a standard is not enough. So Microsoft decided to extend the standard with some “enhancements” they created (known as Simple List Extensions). The fact that they are extending the standard is not something I have a problem with, even though it sounds like the old “embrace and extend” approach they took to HTML. However, what I have a problem with is what they decided to tackle.

Lists and RSS

The new proposed specification allows the ability to create lists. Yes, lists are a good idea if you want to use RSS for something other than distributing content. However, it’s a problem that’s already been solved, and one that has pained much of the RSS community. Let me explain.

In early 2000, when RSS was still in its infancy (version 0.92), a debate erupted on several mailing lists as to how RSS should evolve. The two sides to the debate were as follows:

• On one side were the hard core geeks, who believed that RSS should be reformulated as an RDF specification, tying it into the Semantic Web. Realize that, at the time, the concepts behind this were faily revolutionary: machine intelligence, etc…
• On the other side were the hard core hackers, who believed that the beauty of RSS would lie in its simplicity, and that its adoption would go along the same course as HTML if it were kept simple.

This ended up with two different formats: RSS 1.0 (which met the requirement of RDF integration) and RSS 2.0 (which met the requirement of simplicity).

Well, the irony is that it turned out both sides were correct: On the one hand, plugging RSS into a more formal structure, using things like namespaces and an orderly model could allow it to do more; on the other hand, keeping it simple allowed it to thrive.

Fast forward to today. RSS 2.0 is widely adopted, thanks to its simplicity. And Microsoft is announcing the use of extensions to create lists. RSS 1.0 also enjoys wide support (though nowhere near as wide as RSS 2.0) and supports lists natively. See the humor here: by endorsing RSS 2.0 and creating extra extensions, Microsoft has essentially added a feature that existed in RSS 1.0.

But wait! It gets better. The proponent for the RSS 2.0 specification was Dave Winer, who wrote the RSS 2.0 specifications and maintained it for a fair amount of time after that. However, Dave is pushing a new list and outline format called OPML and is pushing it as the next format he wants people to try out.

So we now have three different ways to create lists. And that’s not even considering the fact that you could use the Microformat concept and had a rel="list" to an HTML element and end up with another format.So Microsoft gets an A for embracing RSS, another A for using namespaces (instead of creating a new version of RSS) and releasing their extension under a Creative Commons license, but gets an F for poor research in terms of introducing a new format. There were a number of other useful things they could have introduced as part of this effort but just generating lists is attempting to reinvent the wheel without really providing any added value.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Microsoft Loves RSS. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

You are hitting on the right points but there may be a need here for more details related to investigative journalism. In the past, reporters were given more of a chance to spend more time on a story. In today’s world of deadline every minute and producing volumes of copy to feed the paper, site, syndication engines, there is more of an emphasis on getting the story out, and getting it out before one’s competitors.

In the process, investigation is dropped. As the deadline every minute frame of mind becomes more common, less time is spent on doing more research. If Watergate happened today, I fear that the story would end up getting buried and Woodward and Bernstein would be redeployed on other subjects. In a way, the Watergate scandal and the Trent Lott ouster parallel each others. A small, apparently insignificant event builds over the course of time and becomes a significant issue that eventually topples a powerful politician.

The only difference between the two is in who did the research. In the case of Watergate, the Washington Post editor gave its reporters enough leeway to investigate further. In the case of Trent Lott, a group of webloggers decided to do their own investigation and share the bits they had found. In a way, the Lott story could be a good example of collaborative journalism and could bring forth a rebirth of investigative journalism. Investigative journalism is time consuming, very costly (think of the output of an investigative reporter vs. a beat reporter) and can be risky from a legal standpoint (most investigative stories uncover things that people don’t want known).

On the other hand, collaborative journalism can spread the load. It allows for multiple people to build on research from previous people. I would not be surprised if, a few years from now, every serious newsroom has an internal blog with section broken out for bigger stories, allowing for multiple reporters to work jointly on stories.

Ultimately, blogging is about community, it’s about people sharing knowledge, and building on other people’s knowledge. This is why the Lott story happened. Someone posted a note about the original remark, a reader remember another fact related to the story and contributed it to the conversation. That triggered an answer by a third person, and so on and so forth. Because bloggers are passionate about what they are doing, they provide heavy background in the form of links to actual sources, making it more and more difficult for anyone to hide in plain sight.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title A response to Dan Gillmor. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

At the item level

: This would allow us to specify a particular date for an item. I think it would be nice for those of us who have several days’ worth of content in their RDF channel.

At the channel level:

These could be encapsulated in to an section that would include all links to outside of the channel.

: Much like points to the page the channel is for could point to a page of information about this channel. this could link to a FAQ or more information about the channel.

: Points to a page where wireless devices can go.

: Points to a page where broadband devices can go.

: Points to a page where narrowband devices (browsers for blind people, text-only browsers, etc..) can go.

: Points to a P3P page to check the privacy rules.

: Points to either a VXML source file (which can be read by a VXML browser) or a sound file. For example, it could serve up a radio feed related to this story.

: Same as above with video or SMIL file.

That said, here’s what a source could look like (changes are bold and URLs are fictional (but I cut and pasted my .91 channel content for speed reasons)):

Suggested RSS .92 code starts here

NewDocTypeLinkGoesHere
>

92
">

foo@email.com
            Copyright 1999-present, Me.

            My channel description>/description>
            en-us

http://www.tnl.net/images/TNLpalmlogo.gif

            125
            44
            My channel description


http://www.tnl.net

http://www.tnl.net/about/

http://wap.tnl.net

http://www.tnl.net/100MBpage.html

http://www.tnl.net/under1kpage.html

http://www.tnl.net/p3p.xml

http://www.tnl.net/myvoicebasedchannel.vxml

http://www.tnl.net/myvideofeed.smil

http://www.tnl.net/newsletter/anewstory.html

            Story 1 is described

10/13/2000

http://www.tnl.net/newsletter/olderstory.html

            Happy New Year

01/01/2000

More Details

As part of the deal, I’d also move the original channel link and image link into the external field under a single link header (unless some people can tell me where they have a different link for the image and the channel.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Suggestions for RSS .92 spec. You can follow him on twitter here or receive his weekly newsletter by subscribing here.