Recently, though, TechMeme has gotten me thinking about the tech blogosphere conversations as a whole and their longer term relevance. To the small “web 2.0″ community, TechMeme serves as a bit of a paper of record; The subhead even claims that it represents the “Tech Web, page A1”, claiming to bring us the important stories. But how do those stories fare over time? Is today’s hot topic a step in understanding a longer term trend or is it just a temporary distraction that will be forgotten a month/3 months/6 months/a year from now.

Fortunately, Gabe Rivera, the founder of TechMeme must have anticipated such a question and provided a way to look at TechMeme as it was a particular point in its short history. Using the simple interface, it’s easy to see the page as it existed at a precise point in time. So I decided to start looking at the site at the same time in single month spaces. The middle of the night and middle of the day position ought to be good time stamps so I decided to look at the site at 12am and 12pm on the selected date. I also had to discount the fact that April 1st is April fool’s day so I could not use the first of the month as this fact could skew the data. Here are the dates and times I ended up with:

• Today: June 2nd 2008 at 12am and 12pm
• A week ago: May 26, 2008 at 12am and 12pm
• Two weeks ago: May 19, 2008 at 12am and 12pm
• One month ago: May 2, 2008 at 12am and 12pm
• Two months ago: April 2, 2008 at 12am and 12pm
• Three months ago: March 2, 2008 at 12am and 12pm
• Six months ago: December 2, 2007 at 12am and 12pm
• Nine months ago: September 2, 2007 at 12am and 12pm
• One Year ago: June 2 2007 at 12am and 12pm
• Two years ago: June 2, 2006 at 12am and 12pm

With 20 data points, here’s what I discovered.

Today

Based on today’s news at noon, it looks like the important subjects at noon in the blogosphere are Adobe’s latest move, combining Flash and Acrobat with their entry in the already crowded (Google, Microsoft, Zoho, etc..) web-based office suite market. At midnight, things were a little less exciting, with discussion around the privacy issues Google Maps is raising with their StreetView offering.

Of course, it’s still too early to tell whether those stories will have a long term impact so let’s roll the tape back a little.

One Week Ago: May 26, 2008

At noon, a week ago, the top story was about a new type of SSD, developed by Samsung. Since it’s hardware, I assume that the impact of this news can’t be felt initially but there could be longer term repercussions. Also of note on that page is a small item lower on the page about Paypal outages. An interesting trend in my research on this is that this story is slowly developing over a period of weeks and months and the noise level appears to be increasing on it.

At midnight, the discussion was around Google’s power and the needed for another organization to work as a counter balance to that powerful force in the search engine space. Coupled with the discussions last night about privacy issues relating to Google maps, it seems we are seeing an emerging pattern here.

Two Weeks Ago: May 19, 2008

Two weeks ago, at noontime, the claim that Microsoft would eventually buy Facebook and keep it close was dominating TechMeme. At this point, no announcement has been made so this is largely conjecture and, while an interesting opinion, it’s not really news. This editorial was largely in response to the news item that dominated the previous 12 hour cycle about Microsoft’s statements regarding pursuing a possible deals other than a full acquisition with Yahoo!

One Month Ago: May 2, 2008

On May 2, 2008 at noon, the big news was… that the Google RSS reader is now available for the iphone. I’m sure many people consider this event as a major turning point when… well, hmm… a big big deal. Amusingly, Adobe was also in the news that day, with news that its flash plugin would escape computers and appear in set top boxes and mobile phones.

Another big subject was Steve Ballmer’s mention that Microsoft could go it alone without Yahoo, a discussion that dominated the midnight page on that day. The Yahoo/Microsoft chat has been kind of the soap opera of our industry and this latest installment was remembered as a turning point (or not) by many.

A possibly interesting trend piece, around midnight, was also intriguing: Will Grand Theft Auto IV hurt Iron Man opening weekend sales. I haven’t seen a follow up on that piece yet, which could tell us whether video games are displacing movies as the primary form of entertainment but my guess is that the answer is no.

Two Months Ago: April 2, 2008

On April 2, 2008 at noon, the top story on techmeme was about Intel’s plan for chips that would power up more mobile devices. Interestingly, this story was largely driven by mainstream media as the lead was taken by john Markoff of the New York Times, followed by comments from Forbes magazine, and Infoworld. The other related story was the press release itself, which can be seen as bloggers pointing straight to the source of the news. I suspect that this story will probably have more legs moving forward. A cursory glance provides glances at developing stories ranging from the rumor stage (that all important Google/Skype partnership or acquisition… which didn’t happen) to the focus on process (like the approval of Office Open XML as an ISO standard).

The departure of Google’s CIO dominated the prior night’s news cycle and word of Apple’s 3G iphone started to filter through.

Three Months Ago: March 2, 2008

March 2, 2008 at noon provides us perspective on today’s news, thanks to Microsoft’s announcement of ITS entry into the web-based office suite market. When put side by side with today’s announcement by Adobe, it seems to start pointing to more of a trend. Beyond that, little news that seems to be of note from a memorable standpoint.

The interesting thing here is that the same subject was leading the previous night’s news cycle. This seems to establish a first rule for techmeme: subjects that survive on the front page more than 12 hours may be worth paying attention to.

Six Month Ago: December 2, 2007

There’s an all saying in journalism that 3 items make for a trend. In the case of this study, it looks like Web-based office suite are definitely the hottest trend around, as the top news on December 2, 2007 at noon was information about the future of Google’s offering in that space (either that or there is an unwritten rule in the technology field that information about web-based office suites MUST be introduced on the second day of the month or wait until the following month).

The subject was starting to climb the chart 12 hours earlier, even thought the discussion at the time was dominated by a Facebook misstep (remember Facebook Beacons? Well, that was around that time). From an interface standpoint, it also brings up something that I’d like to recommend to Gabe: could you add and up or down arrow to highlight if a subject is getting more play or not. On something like this, it would be nice to get an idea of the stickiness of a topic. It appears many topic appear low on the page and move up over time, the quicker and faster they move up seems to indicate the importance of the story and it would be a nice addition to have that info on the screen.

Nine Month Ago: September 2, 2007

September 2, 2007 was a quiet news day. I guess everyone was mourning the death of the newspaper, which was forced by Google on that day, according to the noon-time headlines. There doesn’t seem to have been any other major news around midnight either. This, however, could be an artifact in the data as September 2, 2007 was a Sunday, which is generally a pretty quiet news day as most people don’t work on Sunday.

Interestingly, a story that is just now starting to get more notice is the continuing brushfires around Paypal’s outages. Not that sexy a subject but one that started to be raised around that time. At the time, discussion of Google’s entry in the mobile market centered around the idea they would deliver a device instead of a platform.

Last Year and Two Years Ago

A year ago, at noon, the Techmeme conversation was around porn. During the night, though, the conversation was centering around the acquisition of Feedburner by Google. This is probably remembered by people in the industry as an important milestone and here, techmeme shines at organizing a package with the appropriate conversations.

Things do not improve much if you go further back: 2 years ago, at noon, and midnight, gives us little to mull over.

Conclusion

The data seems to point that the front page of TechMeme largely represents what’s hot right now but does not necessarily highlight stories which have a longer term type of impact. In that sense, it may also be highlighting that discussions in the tech blogosphere are largely centered on insider-type minutia while failing to put things in a larger context. This appears to present a myopic view of the tech world that leaves us with lots of data but preciously little information. So while TechMeme provides a useful tool in terms of getting an idea of the pulse of the conversation “right now,” it does little in providing data that would allow someone to understand the larger trends that are affecting our world as a result of the internet (and web 2.0 revolution).

I would argue that the answer to the question I posed in the title for that post is a resounding yes. Because it deals largely with the trivial and assess little value to longer type impact, TechMeme creates a self-imposed myopia on its readers and participants. A possible exception is when a story manages to survives through multiple 12-hour instances, providing many angles to the same events. But those events are few and far between.

Whether the lack of headlines with a major impact is a phenomenon that is unique to TechMeme or to the tech world in general is a question I’d like to leave to readers and I’d appreciate comments as to your thinking around this.

But all this comes down to a simple fact: if you’ve missed what happened on TechMeme in the last XX hours, days or weeks, you may not necessarily have missed much. so kick back, relax, step away from the computer and, if you need to catch up, you can always pick up a mainstream publication that may cover a distilled version of what happened if it’s of any particular significance.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Is Techmeme myopic?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Looking Back

In the ensuing 6 years, I’ve been going over and over what went wrong and discovered more lessons along the way: the market conditions were wrong, we were young and arrogant, and, for the most part, we didn’t really understand the magnitude of what we were trying to accomplish: to remind people, our goal was to launch a website in 16 countries (15 EU countries + the US) on day one, localizing our site for each of them. At the time (1999), no one had accomplished that broad a coverage (nor had anyone even tried to).

So it seemed a little crazy but, then again, crazy people had built Netscape, Yahoo, Ebay, and Amazon in the previous few years. So crazy seemed not only possible but it seemed to be the key to success on the Internet. The problems we encountered fell in a number of areas: currency exchanges, tax issues, language localization, integration with many fulfillment partners and a front-end experience that called for broadband connections. We basically wanted to build eCommerce 2.0 long before there was a web 2.0.

Looking Forward

So fast-forward to now. Broadband uptake is nearing 50% in many of the target countries and the number of users has grown tremendously, governments have learned about internet ecommerce and now have specific rules relating to it. And integration across many system is what web services and mash-ups are all about. Do I smell progress? So let’s revisit my old post (which later was published in Business 2.0) points and look at them through the 2006 lens.

The Currency Problem

Back then, the 16 countries we targeted meant 16 different currencies.

Today, with the rise of the Euro as a unifying currency, the same 16 countries only have 4 different currencies (the UK still being stuck on the pound sterling and Denmark keeping its currency a national one pegged to the Euro. The US and the Euro are the other two currencies covered.) This greatly reduces the complexity of pricing models across Europe and makes the overall cost of managing the catalog much lower.

Back then, we actually had to build our own currency tracker, with people inputing the exchange rates daily into the system to keep everything aligned.

Today, you can get access to currency exchanges via web services (just off the top of my head, I can think of Reuters and CBS Marketwatch providing this type of data), therefore automating what was once a manual task and, once again, reducing administration costs for the catalog.

Tax Issues

Back then, there was no consistency in the way taxes were assessed on goods sold online. The financial people at Boo.com version 1 spent a lot of time with a big 5 accountant group and a lot of local government to lobby for normalization of rules around taxes on cross-border business.

Today, because all of those governments understand the value of internet commerce and because many have worked in conjunctions with each other (through the G8 and the EU) to normalize rules surrounding taxation of goods sold on the Internet the problem is easier to solve.

Back then, we had to build our own systems to track all the vagaries of the different tax systems. It wasn’t a build vs. buy decision because there were no packages offered on the market to deal with this.

Today, you can buy software packages that has all the taxation rules built in so that problem is no longer one you need to build for. You can just buy the technology and let the vendor worry about the changes in taxation laws.

Language Localization

When we set out to build Boo.com, a strong component was the idea of offering the online store in the local language of the user. Boo.com was actually the first store to offer as high a level of customization by market and we had to make a number of changes to the e-commerce software package to make it into a globalized platform. Remember that, at the time, e-commerce was primarily the domain of US and UK companies so selling in a language other than English was rare. E-commerce sites which sold goods in non-English markets were generally customized on a one off basis but no one, prior to Boo.com, had attempted to have a single back-end system run multiple countries.

Today, more vendors are selling solutions which can be customized across a variety of western languages. The solutions are not yet perfect but, for the most part, they work (there are still a number of issues when it comes to localization across 2-byte languages, especially when it comes to site with mixed languages.) Back then, we also had to develop a content management system that could handle translation workflows and management of content in multiple languages. It wasn’t pretty but it worked and it required a lot of internal translation to happen. Each product had description, sizes, etc… available in multiple languages. That part was actually a fairly large management of content nightmare. Today, modern content management system can handle more complex workflows (allowing to track when translations are completed) and even can provide hooks to farm-out translation of the content to external parties. This substantially reduces the cost of a multi-country offering.

Integration with fulfillment partners

Back then, a fair number of people at Boo.com were experts in EDI (or electronic data infrastructure) because EDI bridges were the only way to integrate into our fulfillment partners. Web services didn’t exist so we had batch jobs triggering every hour to the warehouses at DeutchePost and UPS so they could pick, pack and ship the orders. This was expensive and probably the area where we lost the most money on a single transaction.

Today, services like fulfillment by Amazon provide the same service at a substantially lower cost and with less integration headaches as web services are making it easy to integrate their services into an e-commerce operation. That saving alone could justify the existence of Boo.com 2.0 (actually, it would be 3.0 as FashionMall tried to resurrect Boo.com once already).

Front-end

No discussion of Boo.com can be full unless we talk about its front-end.

The Broadband Penetration Problem 

Many people laughed at the attempt we made at creating a more user friendly interface to e-commerce. Back then, a more interactive experience meant using Flash. It was the only way to get a lot of parts moving together. Things like Zoom-In/Zoom-out or Rotate type of effects were hard to accomplish with DHTML and much easier to do so with Flash. Since XML didn’t exist, we didn’t have AJAX. Since we didn’t have AJAX, we went with Flash. Since we went with Flash, the assets were large. Since the assets were large and the average user was connecting via a 56k modem, the site looked slow.

The idea was that every click should feel snappy, a model now common with AJAX-based applications but we failed in one assumption, which is that broadband penetration would move at a faster rate. Our expectation were that 1Megabit lines (much slower than what one now gets via cable or DSL) would be readily available within a year. That was a very flawed assumption and we had not planned any contingency for any slower a deployment.

Selling clothes requires details

Another interesting challenge was that we were trying to sell clothes online. Evaluating a DVD, CD, or book online is easy. However, clothing is different: when people shop for clothes, they like to feel the fabric, look at the details in the fabric. That experience was hard to reproduce online. Back then, what we set out to do, in order to help mimic some of the experience was to have highly detailed pictures of the goods. 

Every product was shot multiple times at a stunning 5 megapixels per picture (the highest possible resolution at the time). This meant picture files that were about 1–2 Mb per file, something that seems small in the era of Flickr and YouTube but was massive in the era of 56k modems. The advantage of such detailed pictures was that you could zoom in to a level higher than what you could do in a store (part of our attempt to compensate for the fact that you couldn’t touch the merchandise). Today, such level of detail is standard among most of the online clothing manufacturers and with more broadband lines, it’s no big deal.

Another innovation we introduced was the presentation of products in 3D. You could basically rotate every product in our inventory any way you wanted. This, at a time when QuickTimeVR was not on the marketplace. This meant getting our photography partners to come up with completely new approaches to taking product shots, sometimes requiring as many as 15–20 shots per product in order to get everything right. Those pictures were then taken into Flash and adjusted so that you could rotate the product and zoom in and out of it, a feat that now seems pretty standard, using QuickTimeVR.

All that photography work didn’t come cheap, especially when you consider that this was done across 5,000 products and that all the assets were then stored on our servers (Hard Drive space was nowhere near as cheap as it is now). 

Modeling

Another innovation was the introduction of virtual models you could use to try the clothes on. Today, Sears offers a lower quality version of what we were offering back then (their model still requires a reload of the full page to turn it.) Because all the products had 3D equivalent, modeling them was relatively easy and we decided to throw it in as an extra feature that helped enhance the user experience. Once again, because of the processing and bandwidth required to make that happen, the idea was ahead of its time. 

Miss Boo

So we now all know that chatty avatars on web sites are not a good idea. The concept behind Miss Boo was to help make the experience similar to that of a store, with a sales assistant (Miss Boo), helping you out. Our long term goal was to have Miss Boo attached on the back-end to a real person so we could have integrated IM while you were shopping (that plan never came to fruition as the company had other concerns after launch). In the process, though, we’ve learned that avatars are generally despised and probably helped many sites avoid them.

Tagging

Because we wanted the experience to be a more communal one, we had a way for users to tag clothing (well, we didn’t call them tags, we called them “LaBOOls” (labels, with a Boo in the middle, get it?) in the great tradition of badly named things on our site). However, because there was no AJAX or other way to quickly get the data back and forth, it required a reload of the whole page after each tag was applied. The feature was quickly killed in order to gain speed but I can’t think of any other site that had tagging on products at the time (if I’m wrong, please rectify me in the comments).

Chatty Tone

The BooZine (Boo Magazine) was our attempt to create a more friendly, open tone when dealing with users. We didn’t want to be just a store, we wanted to engage the users. When our forums (remember, this is before blogs were popular) started filling up with vitriolic comments, we were forced to shut them down, closing a channel of communication for users to us. It was a real shame but I think our attempt can be mirrored in the way most web 2.0 companies now have a blog that they use to receive feedback from users.

A more mature market

Back then, few people were buying stuff online. Even fewer were buying clothes online and an even smaller number than that was buying hip clothing. Considering all the challenges Boo.com was trying to address, its target market was just too small to make it a successful business.

Today, blogs like CoolHunting, HypeBeast or MocoLoco show that there is a market for the types of goods Boo was trying to sell. That, in itself, could be a good reason for Boo.com to come back: The market they were addressing is finally there. However, it may also be a reason for it to not comeback: the market they were addressing now has competitors in it.

Was Boo.com the first Web 2.0 company?

I have to admit that I’ve been feeling a certain level of uneasiness about Web 2.0: to me, there didn’t seem to be much there that I had not seen before: web services (yup, done since 2000), user generated content (tried it in a limited fashion with with the “labools” and forums), more transparency (tried that with forums in the past), chatty tone (attempted at Boo). What I failed to realize is that where we failed was in the way we implemented things. But looking back now, the reason it didn’t feel new was that much of that experimentation was on our site only, not part of a more widespread phenomenon.

Another thing that got me thinking along the way of Boo.com as a Web 2.0 company was the excellent post on Pixel Acres about the visual design of web 2.0. Let me explain, picking points from the article:

Integral to Web 2.0 is harnessing the input of website visitors. Users can generate content for a web service, promote it in a “viral” peer-to-peer fashion, and improve it’s data quality through their opinions and preferences.

Users of Boo could create their model, share it with friends (following the UGC model, I guess). So the input component was there, as was the sharing one.

Most Web 2.0 sites come across as friendly, approachable and small-scale, using subtle design decisions to gain our trust.

Every decision about the front end was to make it appear friendly, chatty and hide as much of the complexity as possible (that’s why so many people thought what we were doing was easy but badly implemented).

Bright, cheerful colors dominate Web 2.0 sites… Bold primary colors suggest a playful, fun attitude and also help to draw attention to important page elements.

One word: orange. The boo.com site had cheerful colors all over the place (sometimes so cheerful that I worried it would be seen as a toy)

Rounded Everything: The friendliness of rounded corners is in keeping with the comfortable, informal tone of many web 2.0 sites… In a great FontShop article analysing the logos of Web 2.0, it was clear that rounded typefaces are all the rage. This smooth approach to type lends a modern playfulness to a company’s visual identity.

Yup, Boo.com was round, very round, even the logo and the fonts. From a visual standpoint, it was much closer to today’s web 2.0 site than the ones it lived among.

Most Web 2.0 sites devote prime real estate to the message that they offer a free service.

Well, we kept pushing our “Free” boozine (Boo Magazine) and looked at it as a way to hook people into coming back again and again to the site.

You won’t find any stock photography of smiling support staff on a Web 2.0 site — that’s a tactic favored by small companies trying to mimic large corporations. Simple icons and screenshots are the order of the day when it comes to imagery on Web 2.0 sites. 3D and beveled icons can lend elegance and polish to a page design that is otherwise fairly stark.

Boo.com was 100% stock photography free. It was all icons and cartoons.

A good Web 2.0 app ought to be lightweight and easy for users to grasp, and clever visual design and copywriting can help remove barriers to entry. Smart use of layout, color, type and copy can go a long way towards easing the pain.

Well, we failed on the lightweight end of things but the design was to be as airy as possible.

As far as Web 2.0 is concerned, bigger is definitely better. Bigger text, that is. Large text is easy on the eye, and coupled with snappy copywriting makes information easy to absorb. And now that accessibility is cool, it’s possible to be a hotshot web designer and use enormous type.

… and back then, people said we didn’t make good use of the real estate because the fonts on our screens were too big. However, note that accessibility was inexistant at Boo.com

The layout of Web 2.0 sites might be described as minimal. With a focus on legibility and ease of use, good use is made of white space. White space allows important information to stand apart, provides rest for the eye, and imparts a sense of calm and order. Generous leading also makes text copy easier for the eye to follow. Some Web 2.0 layouts are so minimal that they verge on boring, but designed well, an uncluttered page can be incredibly tasteful.

Yes, we had a lot of whitespace.

Friendly, informal copywriting allows a more personal relationship with website visitors.

People complained that our content was too informal, actually. I guess taste has changed in the following years.

So, from a visual standpoint, we may have established some of the rules that are now considered good visual rules for Web 2.0 companies. Of course, feature wise, we didn’t have RSS (it had not achieved the level of popularity it now has) and worked largely as a walled garden (all interaction happened on our site) but Boo.com was probably sitting closer to a Web 2.0 sensibility than most companies that existed at the time.

Conclusion

Based on past history, the complexity that existed back then has largely disappeared, making it possible for Boo.com to exist in the web 2.0 world. The market has also evolved to the point where many of the innovations first introduced by Boo.com are now considered mainstream and where many of its barriers to entry seem to have disappeared. This means that Boo.com could have a chance at surviving this round. However, one would have to be careful about overspending on advertising (a crime that Boo.com was responsible of, with its massive multi-country ad budget). A question that remains on the viability of the brand is whether the errors of the past have damaged the brand to a point where it would not be able to come back. It is probably the most dangerous factor in the rebirth of Boo.com and, if the negative press of the past overshadows the re-emergence of this company, it could be a fatal flaw that could ultimately make this a bad idea.

I wish much luck to the parties involved in the relaunch. Hopefully, they won’t suffer from the same arrogance we suffered from in the first iteration of the company and will be able to build a strong business around this brand.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Why the Boo.comeback makes sense. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Looking at the history

In order to look forward, it always pays to look back. The dominant standards for the web today are undeniably HTML (or its variances like XHTML) and HTTP. More recently, XML has emerged and, increasingly, RSS is becoming the dominant type of XML for sharing a variety of data.

How did each of those standards become a standard. It is obvious now (hindsight is always 20/20) that standards bodies have relatively little bearing when it comes to influencing the succes of a format. Take, for example, SGML, which was the dominant standardized format for document formatting. It was quickly superceded by HTML which, at the time, was not considered a standard.

The same is true of RSS and other standards for syndication. Formats like ICE, CDF, and NewsML were touted as the future when they were first introduced. However, they’ve recently been superceded by RSS.

And even within the RSS world, formats like RSS 1.0, which was supposed to be more semantically sound, and ATOM, which was supposed to be more forward thinking that RSS 2.0, have been losing the war to RSS 2.0.

Bootstrapping is a social phenomenon

What Dave Winer understood, when he sheperded RSS 2.0 into becoming the dominant mean of delivering syndicated content is that the life and death of a new format is predicated on its widespread adoption. And, in order to increase adoption, one has to make something generic, easy to understand, and simple.

Many of the people in the early days of the syndication space failed to see it as Dave did. We believed that a semantically sound format was better and we were wrong. Purity, it turns out is not always a good thing, especially if it gets in the way of people implementing something.

The same is true of HTML. I’d venture that, from a development standpoint, the biggest boost to HTML was a single menu feature that appeared in early browsers and remains there to this day: view source. In the early days of the web, countless developers learned how to do cool things with HTML by reading the source of pages designed by other people.

In a recent issue of ambidextrous magazine, Jeffrey Schox talks about the three stages of technological development: appropriation, early innovation, and sustainable innovation. Here’s how he describes the appropriation stage:

an issued pattent allows innovators to construct roadblocks behind them as they travel down a particular technological path… During the appropriation stage, patent roadblocks waste time and money… The countries, needing to catch up with the designs and technologies of other countries, should focus on collecting revenue and knowledge streams to fuel later stages of technology development.

While he focuses on hardware and electronics in a globalized marketplace, the same truth can be applied to standards. With few barriers in adopting a new standard and by fostering a culture of appropriation, one can easily establish a base of people who understand a new format. As more people understand it, they start implementing it and, after eventually getting smarter about it, start building on the efforts of previous creators. Eventually, those masses of tinkerers get to a critical point, pushing the new format into areas that were unexpected. Some companies eventually get smart to it and see growth in that area, which triggers them into experimenting with that new format.

Eventually, due to a general agreement among all developers, the format becomes a de facto standard. It does not have to have the imprimatur of a standard body (except for some very late adopters or pockets where such imprint is considered important) and moves forward.

What is interesting is the next stage, the one where standard bodies see the area as hot and decide that they need to play in that field. A good example of that is the ATOM format, which has been enshrined into an IETF approved format, and to date has failed to stop the RSS 2.0 juggernaut.

So what happened?

The amazing thing is how simple the issue is. The reason RSS 2.0 has been winning is that it has developed a following. With every new developer learning RSS 2.0, the format goes stronger and the same is true of every company implementing it. Because it is simple, it’s easy to pick up, which means that new developers can do interesting things with it relatively quickly, giving them a chance to become active members of the community and therefore become hooked on it.

The other issue is in keeping things relatively open, while still maintaining some level of control over the general direction. A successful future standard has to allow people a chance to contribute but, in the end, it also needs some gatekeepers who decide what goes in and what doesn’t. The same truth can be applied to any sofware development cycle: for example, Linux may be a widespread open source phenomenon but the number of people who decide what goes into the core kernel or doesn’t is still relatively limited. The same is true of any successful open source project: some level of centralized decision making and distribution of the work: anyone can contribute but not every contribution makes it into the final product.

I’m now seeing some of the same history repeat itself in the OPML space. It’s a format that is very simple and Dave is working very hard on getting people left and right to support it. It’s the same scenario he’s used to bootstrap the RSS format and to bootstrap concepts like blogging and podcasting into the mainstream. It’s a formula that works: keep it simple to implement, maintain some level of centralized control over the roadmap and then evangelize it left and right until it can no longer be stopped.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Standards as social contracts. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Side by Side

The biggest impact will probably be felt on the low end of the creative space. Here’s a rundown of the upcoming battles to come as a result of this acquisition (this is based on the product sheets on the Macromedia and the Adobe sites) and my predictions on who will win each:

Vector Graphics: Freehand vs. Illustrator

While Freehand got a nice following, I believe this one will end up with Illustrator winning, largely due to its installed base. Expect the Flash integration to come into future versions of Illustrator and the Freehand platform to be de-emphasized

Digital Imaging: Photoshop vs. Fireworks

Once again, advantage Adobe, largely due to the larger feature set and the widespread development community that has brought extensions to it. However, expect the optimize for web section of the program to improve as these will be worked on by the old Fireworks team.

Web Authoring: Dreamweaver vs. GoLive

Advantage Macromedia. While GoLive is a nice product, the Dreamweaver suite has a richer set of operations and is better integrated with other parts of the product suite (Flash, ColdFusion, etc…) and I believe one of the key reasons Adobe is acquiring Macromedia is to gain a strong foothold in the web space. Expect more integration of the Acrobat suite into Dreamweaver.

Fonts: Fontographer vs. the Adobe Type Library

Say goodbye to Fontographer, which just isn’t a player in that space, considering the lead Adobe has.

Web Publishing Systems: Macromedia suites vs. Acrobat server

Winner in this category will be the suite of products offered by Macromedia. I believe this sits at the core of the acquisition and that the Acrobat server suite will be merged into the Macromedia offerings.

XML handling: Flex vs. Framemaker

Initial advantage to Framemaker but short lived, as migration path moves it to Flex-based approach. Once again, web trumps legacy. However, one eventual victim of this is Director, which ends up being killed in the process (and I would note hold my breath for the RoboHelp, RoboDemo, and RoboInfo suites to survive for long either.)

Formats: PDF vs. Flash

Tie and win for both. I think both formats will evolve but both will survive this. Tying the two is obviously part of the strategy. What I suspect is that we will see an end-to-end product offering integration of the whole thing from paper to web and back

Monopoly?

One could argue that the merger will create a monopoly situation in the creative space. However, I think this deal will pass in the US and Europe as it will be presented as Adobe+Macromedia vs. Microsoft, Apple, and others. A careful dissection shows that it’s not the case (as it will give the new company a substantial share of the creative market) and makes me think of an ad I saw for Adobe this weekend: the slogan for the new Adobe Creative Suite is “everything but the idea” and it looks like they are now indeed making good on the promise an offering creative types a marketplace where “everything from Adobe” is the only choice (and before you comment about other offerings, ask yourself, what is their market share vs. Adobe+Macromedia?)

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Adobe acquires Macromedia. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Voice Over IP

VoIP experienced tremendous growth in 2004 but it was just the beginning. This year, much more will happen in that space.

Cable providers will start deploying VoIP services on their networks and phone companies will start bundling VoIP services with their DSL offering as a way to compete. By year end, all major broadband providers, whether they are offering services over cable or DSL lines, will have a VoIP service bundled with their access service.

Unable to compete with the larger telcos, some smaller players in the market will merge on order to lower their cost per subscriber by bringing their infrastructures together. Also, independent VoIP companies will sign peering agreement with each other in order to bypass traditional telcos and lower the cost of connectivity from one independent VoIP company to another.

Further pressure will be put on all players on the American market as overseas companies will start targeting U.S. customers. Before year-end, at least one company will offer an unlimited calling to several countries plan. Other plans will provide unlimited calling to each continent. This will start putting pressure on established government monopolies in several countries, especially in Europe.

VoIP will also experience strong growth within the enterprise, with companies looking to open-source solutions like Asterisk to replace their PBX infrastructure with a lower cost alternative.

As all this happens, equipment will not only become cheaper but will also become much easier to use and install. Along with it, new sets will come out, with cordless VoIP offerings becoming much more common. Competition in this space will be on features available in new handsets.

With substantial portions of the phone network switching to VoIP, video telephony will start taking hold. However, the price of equipment will still be too high for those services to experience the kind of growth other sectors in the VoIP market will experience.

Entertainment Convergence

The convergence of the computer and other entertainment forms (television, radio, gaming, mobile phones) will continue, further blurring the lines in the convergence world.

With broadband now being the major way to access the Internet in the United States, Internet usage for new forms of entertainment will grow. Along with it, however, will be a continuing challenge to the established media order.

The challenges faced by the music industry with the introduction of Napster will now be the new reality for the movie and television industry. Five years ago, I started seeing the phenomenon emerge and believe the four step process of the digital asset dance will be full blown for the MPAA this year. The MPAA will spend part of the year suing companies and users for downloading movies. However, they are also better prepared that the music industry in that they are already offering legal download services like MovieLink.

While litigation will be one of the ways convergence appears on the front page, many providers will find a way to mine this new world for new dollars. Expect some companies to start offering legal download of television programs for a fee. As the Internet becomes the standard telecommunication infrastructure, content will start getting carried more heavily. Phone companies will start using this to offer bundle TV services with their DSL offering as a way to compete with the cable TV companies that have invaded the telecom turf. Before year end, at least one traditional telco will offer TV over IP. All that content will be protected by DRM systems, getting people more and more used to having less and less rights over the content they receive.

Meanwhile, on the wireless end, the introduction of more powerful mobile phones and the introduction of faster mobile phone networks will also play out in the favor of content producers. As voice traffic revenues continue to decrease, expect mobile phone companies to push data services such as downloadable movies and downloadable music more heavily. By year, MP3 will be the standard format for cellphones and Apple will offer a mobile phone version of the iTunes music store, allowing users to download music from the store and customize their phone with the latest hits.

On the non-Internet end of things, video on demand will continue the strong growth it experienced in 2004 and more programming will be offered in HDTV format, prompting an increase in sales of televisions and tuners that can receive those signals. Meanwhile, radio will follow the path taken by cable television in the early 80s. As satellite radio takes hold as the new “edgier” alternative to traditional radio, people will get more used to the idea of paying for radio. However, they will also require that those services be offered over the Internet as well as over the proprietary networks like XM and Sirius.

But not all content will be coming from big corporations. The grassroots will also play a key role in the distribution of online media in 2005. While podcasting has been the domain of a few geeks in 2004, easier to use tool will bring the phenomenon to the forefront and expect more audio services to be available from regular users. Following on the tail of this phenomenon will be an increase in videocasting from individuals. Much of it will be disappointing but a few gems will emerge, creating new stars who will emerge from the Internet and move on to more traditional media, based on the fame of their online offerings.

Business

Mergers and acquisitions will dominate the software world this year, as more companies realize that the only way into the enterprise is through a complete set of offerings. Expect several multi-billion dollar mergers and/or acquisition. In my mind, McAfee will be acquired or merge with either Symantec or CA; SAP will be acquired by Microsoft; Business Objects will be acquired by Oracle. As holds true for such precise predictions, none of this will actually happen the way I predicted it.

In late 2004, IBM left the personal computer business, selling its unit to Lenovo, a Chinese manufacturer. Expect the same to happen to at least one other PC vendor this year as the margins on personal computers continue to decrease, turning them into commodities.

Apple

Apple, which to date has resisted the price pressures other computer manufacturers have experienced, will introduce a cheaper version of their Macintosh. This, however, will not stem the continuing loss of market share they are experiencing. As Linux continues to grow, the Apple story in the computer business becomes more and more difficult and the company will increasingly rely on the consumer device business as its savior, building a new economy around the success of the iPod and iTunes music store.

The company will not, however, release a video player this year. Among some of the new features I would envision coming from Apple are:

• A flash-based iPod, which will be even smaller than the iPod mini and will be in the $100-$150 price range
• A partnership with a phone company to create a phone that will be able to download music from a special version of the iTunes music store and play MP3 ringtones
• An iPod with audio recording built-in
• A portable camera with iPod-like features
• A new way to send pictures from the iPod directly to printer via Airport express

While it focuses on the music business, Apple will not spend much time updating its laptop business. Adoption will drop in that part of the business as PC vendors start selling sub-$500 laptop PCs, making the iBook look expensive by comparison. Apple will try to enter the low cost market but not with a laptop: they will introduce a mac without monitor for under $500, offering integration with the iPod, and plugs to attach the computer to a television as its major features.

On the software end, the company will introduce a Word Processor and Spreadsheet program. They will release them, along with Keynote, as a complete package named iWork which will be aimed at students and small businesses. The package will be available for free on new computers.

Development

Blogs and RSS will continue their growth and will move strongly within the enterprise space. Adoption of RSS will continue its explosive growth but crest in 2005 as users start trying to find ways to cope with the information overload. New components in RSS readers will attempt to help organize RSS feeds but those basic efforts will initially fail and discussions will be set towards the end of the year as to the effective way to organize large amounts of data.

Weblogs and content management systems will start covering some of the same ground and enterprise will start using weblogs internally at the departmental level. Meanwhile, external employee weblogs will start becoming the focus of more litigations as corporation try to retain their intellectual property and fight the kind of transparency that comes from having employees talk openly on the web. Internal rules and regulations will be set in how employees can use blogs.

Meanwhile, in the development world, Service Oriented Architectures will continue being the approach to delivering next generation services. SOA will grow largely internally but some companies will start exposing some web services via XML to their partners. A new set of interesting new applications will come out as a result of those exposures.

Security and trust will continue to be big subjects and I suspect that trust will become an even bigger one with new standards emerging around the concept but no general agreement as to the best implementation.

Open source software will continue its strong growth, getting into more and more specialized fields. With the delays in delivery of Microsoft’s next operating system, Linux will continue to grow but complaints about price will start to arise. While the open source movement has offered free software, there will continue to be an increase in the price of supported version of the software.

Personal

I’ll promise to update the blog more often, will do OK for a little while and will then fall back into my regular pattern of a couple of updates a week. Or not… Either way, only the new year will tell.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title 2005 Predictions. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

First of all, let me say that while it is an interesting package, it is not without flaws. For starters, the lack of indicator when the Spam folder and/or trash have stuff in them is a bit disturbing. Sure, it’s not that much of an issue when you have a gigabyte of space but, for those of us who are particularly clean when it comes to online operations (I generally try to keep my virtual trash relatively empty), it’s a bother.

On the spam blocking end, Gmail does an OK job but is nowhere near as good as simply using Squirrelmail with SpamAssassin, or using the spamcop black-list. On the good side of this is the fact that, in the course of a day of use (about 600 messages, most of them spam), Gmail had no false positives in terms of tagging things as spam. On the bad side, it failed to tag hundreds of messages as spam. I’m assuming that Gmail has a learning curve, like other email packages, so I wouldn’t fault it yet on this but, looking at other packages, it does not pass the test in terms of being an efficient spam-blocker.

The concept of conversations is an interesting one and I can easily see how this kind of interface paradigm represents the next step in the evolution of email interfaces. It is clean, simple, and adds tremendous value, especially when tracking discussions across mailing lists. Combined with labeling, it could become a good way to manage mail.

Speaking of labeling, I was disappointed to see that every non-spam email lands in the inbox. If I set a label and/or rule against a piece of email, I would like to see that view move to a different view/folder than my inbox. Once again, under the edict of keeping things simple and clean, I generally prefer to have a relatively uncluttered inbox, with emails being filtered left and right into different folders/views.

Another thing that annoyed me with the interface was its heavy reliance on JavaScript. There should be a Gmail Light version which does not require those tricks. This was most painful when I tried to access Gmail via my Treo 600. Since I don’t always sit in front of a full fledged PC, I can’t always use JavaScript and this means that, until Gmail offers either a way to access mail via pop3/imap or a non-JavaScript browser, my Gmail account will not be the primary one. The other problem with this is that certain browsers, like the popular Safari browser, are not supported. That’s a problem for those of us that do not believe in staying with one platform and/or browser. I tend to use Mozilla when on a PC (kudos to the Gmail team for their support of that browser), Safari on the mac, and Blazer on the Treo and in this day and age, I expect web sites to work on all three.

All and all, Gmail is an interesting email package but I expected more. Maybe it was the hype; maybe it was because it’s a Google product. Either way, I’ll be following its progress over time.

moving forward, I’d like to see the Gmail team develop an API (using SOAP or XML-RPC) so a developer community could start adding features or building on top of Gmail. It seems that this is another area that Google should investigate.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title The day I broke Gmail. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Considering the fact that Google already has both an XML-RPC and a SOAP bridge, it seems that this kind of service would be a no-brainer. Here’s how it would work: A message would be sent out in XML to either of the service and would include the title of the page, its URL, a status code (either new, updated or deleted), and specify the level of crawling you want (page or site). The Google service would then take this info and crawl the specific page or site as required.

The advantage for Google is clear: sites that use this service would no longer be part of the regular crawl for Google spiders, which could help Google focus on spidering other (new) sites that are not on this list. For sites that are running Google ads, this would also allow Google to produce better targeted results.

The advantage for web site developers is also clear: this system would allow them to more quickly get into the Google index, a pretty good source for traffic.

In the end, it’s a win-win scenario for all involved. So when will we see this new feature from Google?

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Google Ping. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Compression, Glorious Compression. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the meantime, I do believe that both format can co-exist, and that a winner will emerge whether you inject money into the debate or not. Right now, I’m still betting on RSS winning because it has built a strong following. Until aggregators and readers are available for (n)echo, the new standard will largely be an academic exercise.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title It’s all XML in the end. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Discoverability and Security

The first assumption is that software should be able to connect automatically. While this is generally a good idea, there is a need to set up different levels of accessibility. Businesses generally will want some level of controls over how accessible a machine is. From here, one must establish a level of trust to handle relationships between machines. This should include some basic categorization, user being the lowest level of categorization, then raising up to different groupings. For example, a user could be a member of a team, that team could be a subset of a larger division, which itself would be a subset of a company, which would be a subset of an industry. The idea here is to automate the process of identification and still ensure a degree of trust in order to maintain security. While a piece of software can be network aware, the network may not necessarily want it to be aware of all resources. For example, if I am a visitor at BigCo, BigCo may not want me to have full control over all their resources and full access to all their services. My machine should broadcast my credentials and, based on that, have access to certain resources.

Why Buddy Lists are NOT the way to go

Tim advocates the use of buddy lists to set up those relationships. I would venture to say that buddy list do not provide the level of granularity required. From there, there are two potential ways to go: enhance buddy list to allow greater levels of categorizations or come up with a completely new format. I would be tempted to go for the former as it is building on top of an existing standard.

Two way data and XML formatting

Tim makes a point that every piece of software should expose some version of its data as XML feeds. While I generally agree that the data should be represented in a common format, XML being the ideal choice, I object to it being a feed. What applications should provide is an API that gives access to that data instead of a feed. The reason for the semantic disagreement I have here is that a feed is generally pushed or pulled on a regular schedule, no matter whether it is needed or not. Providing an API would ensure that the data is only obtained upon request, therefore conserving precious network resources. A good example of feed misuse was Pointcast, a software client that would poll the network every few hours for feeds. The problem was that it would do so at the same time for every client on the network, thus generating network traffic spikes on a regular basis, and generating much hatred from network administrators.

A proper API could be designed using either XML-RPC or SOAP as a way to carry its messages.

Where does the data go?

The other issue is where the data should reside. As a general rule, computers are no longer well suited as the only repository of data. There is a need to represent data in a fashion that makes it largely independent of the platform it’s running on. A large part of the problem here is who you trust (there’s that trust issue again) with your data. For example, buddy lists in AIM are stored on the  AOL servers. Do you trust them with that data? Would you trust them with more personal data (written documents, etc..) ? Would you trust Microsoft with it? Would you trust anyone?

This brings up interesting possibilities in terms of either keeping the data on a single computing device, from which it might be shared, or moving it in a lot of different places (making it more difficult to ensure change control and general data management). This is an issue that still needs to be resolved.

Online/Offline

The one point that Tim does not cover is the online/offline challenge. One cannot assume that a computer is always connected to the network. As much as we would like it to be that way, computers are often disconnected from a network, whether it is on a plane ride, or when in a place where network resources are limited or inexistent. Programs should be aware of that state and still be able to work properly when offline. As a result, software should have a mode that allows it to check whether network resources are available or not. If they are, it should check the sharing arrangements. If there are none, it should still provide basic functionality.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Tipping the Edge. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

1. familiarity
2. consistency
3. generalizability
4. predictability
5. simplicity

Let’s delve further into each of those in more details.

Familiarity

The concept of familiarity is almost self explanatory. It talks to the way people “expect” things to happen. A good example of this is the web browser. Most people expect the browser to be divided into seven areas: an application menu, which mirrors the look and feel of other applications on their operating system (such as file, edit, view, bookmarks (Microsoft Internet Explorer calls those “Favorites”), Tools, and Help); a loading graphic (which is animated when the browser fetches a page); a navigation area, which includes familiar buttons such as backward, forward, stop, refresh, and home; a web address box where they can type in an address to get to a web site; a bookmark (or “Links” as Internet Explorer calls it) bar, which lists sites that are bookmarked; a main window, where the content of the site they are looking at is displayed; and a status bar, displayed below the main window and providing information on whether things have downloaded or not.

As you can see from the list of what makes a browser screen, this is already a fairly large set of components to think of, when designing a browser. Mozilla, my favorite browser, succeeds in mirroring most of this properly. However, the lack of a home button in the navigation bar of the default install of the software may confuse a lot of users (There is a patch to resolve that problem but beginners will not know this. This creates some concern for the user, which easily unsettles them because, if this part is unfamiliar, what else can they expect?

The challenge here is to start thinking like a low-end user. Mozilla is a very powerful tool but how do you make it easier? As developers, we often forget that the people who could use our software may not know as much as we do (or worse, some developers believe that people *should* be experienced enough to use their code). The best way to handle this when working on designing screens as part of an OSS project is to show it to people who are not programmers (family members, or non-geek friends are useful here!). If it takes a regular user more than a few minutes to understand what the program does, you may have a usability problem.

Consistency

Consistency talks to a certain level of expectation. As a general rule, users expect a program to act in a consistent fashion. Consistency issues arise when a piece of software looks different from area to area. For example, if you let the designers loose on your interface, you may end up with different fonts, font sizes, buttons switching positions, etc… This happens a lot when skinning applications.

In order to maintain consistency, ensure that your application reacts in the same way on its sub elements as it does on the top ones. For example, if you use buttons like OK and Cancel next to each other, make sure that they always show up together so that you don’t end up with an OK or Cancel button standing on its own or with something like Go and Cancel in one screen and OK and Cancel in another.

A consistent interface breeds familiarity, which in turns makes your application more usable.

Generalizability

The concept of generalizability expands on consistency but goes beyond your application. Generalizability talks to the wider world of all applications like yours. As a result, it’s kind of a pack mentality thing. If you are working on building a better mousetrap, you have to make sure that people realize that it is a mousetrap. As a result, you have to use some of the elements and attributes of other mousetraps.

The best advice in terms of maintaining generalizability is to look at what similar applications do. For example, going back to the browser, the edit preference screen is generally organized in subsections. In the case of Netscape and Mozilla, they show up in a category list on the left of the preference screen. In the case of Internet Explorer, they show up on the top as tabs. In the case of Apple Safari, they show up as big icons on the top of the preference screen. As you can see from here, there is some level of consistency in terms of organizing that content.

Another example of generalizability is the tabs implementation in Apple Safari. Mozilla and Opera had taken an early lead in establishing tabs as part of their browser offering. Opening multiple tabs showed a tab navigation bar and opening a new tab on the mac implementation of those browsers was done by using Apple-T. When Apple implemented their solution, they bowed to the consistency rule by implementing tabs in the same way as its predecessors.

Predictability

Predictability is, as expected, building a system that works in the way you expect. This is much tougher than one would think as level of expectation are different based on user levels.

For example, advanced browser options in Internet Explorer are under in an item called “Internet Options” under the “Tools” menu. The assumption here is that anything that is user configured is optional and that, in order to configure it, one would use a tool. On Mozilla, the browser options are in an item called “preferences” under the “Edit” menu. The assumption here is that the user would want to edit their preferences. Two different paths to the same area, which can lead to confusion.

Because Internet Explorer has the leading market share, we must fall back on the generalizability principle here as to where a user should expect those options to be. This points to one of the issues related to all the points brought up so far: sometimes bad interfaces are what the user expects. Trying to change that is often difficult and goes against the concept of learnability.

Simplicity

The last item in learnability is simplicity. The simpler the interface, the easier it is to use. Here’s an example: If I create a button called XML in an application, fellow geeks will expect that once they click on that button, they will see the XML version of the document I’m presenting. However, someone who does not know what XML is will look at that button and be confused. As a result, that button should not figure prominently as part of my default interface.

A good way to enforce simplicity is to provide a regular user and an expert user setting. For example, Apple Safari is a very basic browser when you look at the out of the box version. However, several tools can reveal new menus that will be used by experts. If I were involved in development on the Mozilla interface, I would recommend that it ship with a default dumbed-down set of menus. In the preference setting, one could turn on the super-user mode, which would then provide all the remaining menu items. This would allow to create a simple-looking browser, while retaining all the great features under it for more advanced users.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Usability 101: Learnability. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

This is a bit worrisome as it seems to close up more information on the Apple store. Does that mean that Apple is seeing its seeing as a closed one? If that’s the case, it’s a real shame.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Open or Closed?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

The following basically tells us that we are dealing with an XML document.

It’s encoded in UTF-8, a format that allows to show the largest possible set of characters. This clearly shows that Apple intends the store to be something that they will use beyond the US as they might plan to use foreign characters in their XML definition.

Also of interest here is the note about standalone being no. This means that there is no interest in the document not being part of an application (ie. the store).

Next up is the page definition:

Of interest here is the fact that they differentiate the different pages. This one is the result of a search so the pageType is set to search. I haven’t yet fiddled with other pages but I suspect that they have other potential use for this.

Once you’ve got your document defined, they break it down into three elements:
Path, ScrollView and TrackList. Let’s look at those now.

The Path Element

The Path element basically provides information as to where the document resides. Under the tag, you find a sub-element called PathElement. It shows up as follows:

 /WebObjects/MZSearch.woa/21/wa/advancedSearchResults?songTerm=Morritz%20Discovers%20the%20Red%20Violin

Now what we have here is a page called “Search Results for: Morritz Discovers the Red Violin” which is located at http://phobos.apple.com/WebObjects/MZSearch.woa/21/wa/advancedSearchResults?songTerm=Morritz%20Discovers%20the%20Red%20Violin

Basically, what it gives is the location of the document. Nothing to find here.

The ScrollView Element

This one is a monster of information related to how to display things on the screen. Of interest here is the fact that they position everything using a new form of XML and using outside pointer files for font handling. It looks like Apple is using some WebObject syntax to position their elements on a page. Why they decided to do so is obvious when you realize the NeXT used to be Steve Jobs company and the Apple store is running WebObject.

However, it would have been much nicer if they were to do this as plain XHTML + CSS or XML + XSL. The syntax used in their file is bulky at best and, unfortunately, can only be parsed by using one of their tools right now.

The TrackList Element

This element describes metadata about the particular track (or tracks) you’re looking for. It’s using Apple’s Property List.

This is a great throve of data for anyone who might, in the future, want to develop a service that integrates with the Apple store. Let’s look at it in more details.

The first thing it does is tell you what type of list you’re dealing with:

listType
search

Basically, all it’s doing here is telling you that this is a list that was the result of a search.

priceFormat
$%0.2f

This tells you that the price format is in tenth of a cent. An interesting thing as it seems to open up the potential for microtransaction in the future. Based on the model they seem to be adopting, you could technically go much lower than that, which shows some potential for a future where you might end up paying less than a cent per track. Interesting.

The next key is items. This can list several item. For the purpose of this example, I actually looked for a search result that would return a single item but the same logic applies here whether you have one or many items. The string looks as follows:

Items

… and it is followed by an array that list each item.

So what info do we get about an item in the Apple store?

So here’s the real meat: What makes a track special. Everything here seems to be based on a key-value pair. I’m now building up a table that will show how each relates:

So Apple provides a lot of information in this. For starters, I’d like to compliment them on the breadth of metadata they provide on each track. This is clearly a great value add to anyone who wants properly tagged music. However, it seems that they could offer the tracks in other formats (MP3, or Ogg Vorbis, maybe? ).

The other interesting thing when reading this file is what it reveals as to future potential. It is pretty clear that they plan to offer the system worldwide. It is also interesting that they classify things like media types as it could open up the potential for the Apple movie store in the future. Also of interest was the choice in terms of how small the currency increments are as it could point to Apple eventually looking to sell things for less than a cent. Could we be looking at a rental system in the future? However, there is no information here yet as to what they get from us in terms of data. I’d still like to know that and, if anyone has pointers, would be interested in reading more about it.

Last but not least is why is it that Apple decided to create their own metadata when other solutions (such as the ID3 system used in MP3s) seem to provide similar info? Some of it points to the potential for a proprietary closed system but, by publishing this data in the open, it seems that Apple is willing to take a step in the right direction, that of openness.

My guess is that they will soon offer a web service interface to the store, similar to the one offered by Amazon. It makes sense as it would allow a whole new community of developers to create applications that would drive more money into Apple. However, without a clear understanding as to the DRM setup, it is difficult to estimate what benefit others might get from this.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Apple, XML, and the Music Store. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

As readers of the weblog may have noticed, I’ve been getting into an increasingly obscure area of the Internet by trying to meld two different web formats (RSS and XHTML) and come up with documents that could be understood by multiple devices (web browsers, RSS readers).

The exercise was largely an academic one to study the validity of statements made about the semantic web. Much has been made about the subject and I figured that I would test the validity of the statements being made. The idea behind the semantic web is that the web could become embedded with some basic intelligence, allowing computers to understand extra tagging in a document and allowing for those tags to provide more information.

In order to tie everything together, the W3C came up with RDF. The framework uses a concept called namespaces. The idea of a name space is that you can create shortcut in a document and refer to multiple types of XML and “embed” them all together. This is very good in theory because it frees the framework from actually being smart, leaving those kinds of details to the people that write XML standard. Based on this, you have a set of modules (think of each XML standard as a Lego block) that you can tie together using RDF. Or so the theory goes.…

RSS as an RDF module

While there has not been many example of the semantic web, one area where there has been some development is the syndication space. A few years ago, a new format called RSS (for Really Simple Syndication) was created to syndicate stories on the web. The basic structure was simple: every file was a channel, and every channel had item. An item was a link, a title for that link, and a short description of what it was about. It was nice, it was simple, it was the perfect thing to put together a proof of concept about the semantic web. After many fights within the RSS community, a new RDF-specific version of RSS came up. Now, remember that RDF is supposed to tie all that stuff together so technically an RDF-based RSS feed should be modular. RSS 1.0 (as this new formulation of RSS came to be known) has its own definition that can be referenced in a namespace.

XHTML modularization

In an effort to allow to existing HTML document to bravely move into that new world, the world wide web consortium came up with XHTML, a reformulation of HTML that follows an XML structure that is modular and can be used in the semantic web. XHTML also has its own namespace reference.

Assumptions

One would then think that with two different document types, with two different namespaces could be put together using RDF and work properly. Let’s just review our assumptions so far and look at their logical extension:

IF
— the semantic web is a representation of data on the world wide web,
— and if RSS is data
— and if the world wide web is composed of documents that are XHTML
— and if XHTML is a representation of HTML as XML
— and if XML is modular
— and if the modularity is handled through namespaces

THEN
— purely theoretically, it should be possible to have a document that is composed of 2 modules
— then those two modules would be referenced through namespaces
— then a tool that reads XHTML would use the XHTML tags.
— then a tool that reads RSS would read the RSS tags.
— then a tool that can read both would look at the structure of the document and, based on that, represent the data appropriately.

Cognitive Dissonance

Of course, the theory looks correct but it is when trying to implement this that I started to run into problems. For starters, there is no way to used XHTML as your base document. The W3C in all its wisdom essentially said that you can’t do this. Not ideal but I figured that we could go the other way, embedding an XHTML document within an RDF module. But there comes the next problem. In order for a document to conform to the XHTML standard,

the root element of the document must be

and

there must be a DOCTYPE declaration in the document prior to the root element.

So basically, there is no way to embed it in another document.

Conclusion

Now this sounds like I have reached an impasse. An XHTML document cannot be embedded into an RDF one and an RDF document cannot be embedded within an XHTML one. This means that XHTML cannot be treated as a module (since the root element must always be . If we are to embed documents with any kind of semantic markup, this does not seem to make much sense to me.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Module Madness and Semantic Stupidity. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

The problem arises out of the need for a DTD to be specified as part of the XHTML core structure. The same problem exists in XHTML 2 which could be a problem moving forward. What is needed is for XHTML to chuck this dependency and rely solely on namespaces for the purpose of establishing conventions. After all, if an XHTML 2 document is reformulation of HTML in XML, why would it need a DOCTYPE ? If it continues doing so, it could jeopardize its modularity and will not allow for HTML to converge with other XML formats. Since the goal of the XHTML 2 workgroup has been to reinvent HTML, why did they make that decision? This is a problem that needs to be addressed before the full specification actually comes out.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title No convergence. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

First of all, I would ensure that all the links have proper titles, something that a lot of people tend to forget when designing pages but which can be useful for disabled users. Second, I would replace the validation logos with a much friendlier CSS only alternative, similar to what some have done with the XML button.

Second, I would put the A-to-Z elements in a list, as they should properly be. This would also take care of clearly differentiating them instead of using a CSS trick to hide special characters.

The descriptive text about the consortium is needed on the page and could go above the news section in that design in order to match the existing information available on the page and the proper RDF tags would need to be reinserted in the page to ensure its continued progression with the semantic web.

Last but not least would be a change of color in the masthead to align with the existing color set presented by the original W3C logo (basically changing the site’s primary color from orange to blue).

A very good effort and I hope that the people at the W3C will consider moving to that new design. However, the redesign also points to a big issue with the existing site: not only is its wrong page boring, in comparison to what can be done using the latest standards, but there is also a certain inconsistency in the way sub-pages and sub-section of the site are presented. While the existing front page provides a lot of information, there does not seem to be any overarching design thought in the way each subset is organized, with some pages looking as if they came from the web in 1993, 1995, and late 90s-earlyish 2000s. Maybe the consortium should consider developing a style guide for its overall site. After all, shouldn’t the site be an example of how standards can be implemented and still look professional? On that end, the W3C still falls short and it would be nice to see them doing some work in that direction.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title A matter of Style. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Right now, this page is served to you either in HTML or RSS. Those are two different languages. One, HTML, is understood by your web browser. The way text is bolded, for example, is that I put in a little tag that told the browser to bold this word. The browser reads the tag and presents it appropriately. The other, RSS, is understood by what we call RSS readers. Those are programs that you use to subscribe to a channel. A channel is something that you would receive every day. That way, you don’t have to go and check the site to see if it’s been updated. Your program goes and gets the information. RSS is an XML-based language. What it means is that there is a lot of information in that channel that is there just for the benefit of that channel, to allow to present only the newest news to you.

A couple of years ago, Tim Berners-Lee, the man who created the web, looked at his creation and realized that there was a jumble of pages and that, in order to make sense of it, we needed to give things a little more structure. So RDF was born. What it is, basically, is a way to organize the whole web so that computers could talk to each other without humans in between. This has potential uses and here’s an example I thought up: The smart calendar.

Joe and I are working on a joint project. Joe is in London and I am in New York. I want to arrange a face to face meeting with Joe. Right now, I either call, email, or contact Joe in some way and we figure out a time when we can meet face to face, then agree on a city in which to meet, then make the necessary travel arrangements. What if I went to my calendar, typed in meet face to face with Joe, and my calendar and Joe’s started discussing when the best time and place would be? My calendar would check my availability and Joe’s. It then would check if any of us has any travel plans in each other’s city. Based on those, The two calendars discover that I have a trip to Paris set for next Tuesday to Thursday. As a result, the calendar would recommend that I go to London to meet with Joe on Friday. If both Joe and I agree to this, my computer would then go to the travel reservation system, check prices and flight times, book a flight from Paris to London on Thursday Night, cancel my Friday morning flight from Paris to New York, book a flight from London to New York on Friday Night, cancel my hotel stay in Paris on Thursday night, book a Thursday night hotel stay in a company approved establishment near Joe’s office, and notify Bob (who’s also in London and with whom I had scheduled a conference call) that we can meet face to face when I’m in London instead of doing it on the phone.

Usually, this would have taken several discussions, a whole slew of new flight and hotel reservation changes, and a lot of wasted time. Using a semantic engine, all this would be automated. A lot of computers would have talked together (first mine and Joe’s agreed on time and place, then my computer talked to the travel company with which I had my flight and changed that reservation, then my computer talked to a number of airline companies to see who had the best price on a flight from London to New York, then my computer talked to the hotel reservation system in Paris and canceled one night, then my computer talked to the company computer to see what hotels it approved near Joe’s office. It then talked to computers in the several hotels in London to find a room with my preferences and within my price range. Having done so, my computer talked to Bob’s computer to tell him to change the appointment from a phone conference to a face to face meeting. Bob’s computer talked to a computer in his building to book a conference room. Having done all this, the hotel and airline computers then talked to my company’s accounting systems to agree on billing) because they all talked similar languages (or could point to a translator who would explain how they could. THAT is a practical example that would save money (finding the lowest price on airlines and hotels, reducing the number of trips) and increase productivity (saving time spent on certain tasks) thus allowing me to spend more time on money-making tasks.

All this is still a long way away but if the dream of a semantic web is realized, it will become reality.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Geeks to English. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the discussions surrounding my suggestion of how we can level the playing field, I’ve learned a couple of things:

First, whatever solution we come up with must be easy to implement. It is easy for those of us who are more technical to come up with XML rules and complex structure to represent the world. However, most people neither have interest nor experience in experimenting with such thing. Hence the first rule of any answer is that whatever solution is implemented, it needs to be simple.

Second, trust is a very large issue and some portions of it are being addressed. For example, FOAF allows you to establish trust between friends. But what about concepts (can I trust this person’s opinion on a movie review? can I trust their opinion on an Internet standard?), things (do they own the product? did they buy it or was it given to them for free? If it was given to them for free, who gave it?), people (I think that’s somewhat covered by FOAF but there’s more that needs to be built into it). This is a wide space and needs to be properly categorized for a model to work.

The next question is one of granularity. Should a trust statement apply to a whole site or to a particular entry on that site. As Scott points out “implementing this on a per post level would definitely increase the amount of work to blog content which isn’t good at all.” So what’s the option here? Is there a way to modify editing tools so they have a caveat emptor option?

Another question is how that information is displayed. Should there be a browser plug-in? an RSS reader plug-in? Should it be embedded in a feed or a page? How would the user know when the level of trust on an entry or a site is lower than expected? And, more importantly, how would one be able to check that a “trust statement” can in itself be trusted.

Lots of questions to which I do not have an answer yet but I think that they are worth pondering and I will post further on this soon. (but can you really trust that last statement? )

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Trust, truth and networks. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Here’s how it works:
First, there’s an XML definition to say this is an XML feed:

This is followed by the tag which says this is my full disclosure feed.

Generic information similar to an RSS feed includes the weblog title, weblog URL, the RSS URL (this is important because some people may only read a site through an RSS reader), the type of site (this can be either personal (for a personal site), corporate (for a site run by the corporation pushing the product), or marketing (for marketing agencies, advertising agencies, public relations companies, and other outside agencies promoting a product on behalf of a client), and a contact part (this can either be an email address or the URL of a web form to contact the person or company).

We then go into the section For every type of ad, we include a „ , and . Let me go into more details on this:

Type

: There are a number of ways in which a marketing placement can appear. It can be through the form of a banner, a button, a pop-up ad, a text-ad (similar to Google textads or metafilter’s), a links (within a blogroll, or navigation area, or an entry. If you want to make it clear, you might want to include all of them, or you can pick and choose which ones you will include.

Compensation

: This covers what kind of compensation you received. It can be none (if you just want to say we don’t run this type of ad), goodwill (if you think you’re going to get good karma out of it), cash (the hard stuff), product (if you got a free product out of it) or link (for example if you link to a friend and they link back to you).

Value

: This is a free form field where you can list what is the value of the goods, services, money you received. Some of it could be free traffic, for example.

Entry

: This one is in the off-chance that one day, you did decide to do a one-off deal on a particular entry or if you want to section off particular types of marketing to a particular section. It can be all (for all entries), none (for no entries at all) or a URL (which would be the permalink of the particular entry this applies to).

That’s about it. I just mocked it up in a few minutes so I expect a lot more fleshing out to happen. In that attempt, I invoke the LazyWeb to discuss further.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Marketer, Marketer, where have you been?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the meantime — and possibly as a result– Microsoft and Verisign have just announced a new security procedure for person-to-person SOAP transactions, but a workable mechanism for securing Internet transactions between software and software may be years away.

Some of SOAP’s architects contend that building security into their protocol would only sacrifice its simplicity, and that the HTTP sessions that SOAP transactions rely on can already be secured at the session level, with protocols such as SSL. Moreover, securing sessions from outside interception, security experts believe, cannot protect transactions from two other perceived threats: interception from the inside and bad programming. With a protocol extension to SOAP for message attachments in the works, a third possible threat emerges — one that too many have become familiar with: malicious scripts.

Chris Dix, a SOAP programmer with FMStrategies, sides with the majority in believing that it may now be incumbent upon developers to endow applications with the specific security measures they need to communicate on open networks: If you opened up your [program’s communication] interface to be broad enough to accept things that might be dangerous, Dix says, then it would be your responsibility as a developer to make sure that the requests that might be dangerous came from people who knew what they were doing, and that you built in security.

Unlike the exchange of documents — spreadsheets and word processor files — between two people who can use public key infrastructure (PKI) or other measures to identify each other, distributed software components will communicate with one another without human intervention. In the new net services platforms such as Microsoft’s .Net, Novell’s One Net and Genuity’s Black Rocket, distributed software components will be everywhere, placing remote procedure calls (RPCs) to one another using the XML protocol. So why is the W3C close to deciding that security is not an issue, at least for them?

Should W3C Address Security Concerns?

The security debate began last May, when Ken MacLeod, an engineer of XML-RPC — a SOAP forerunner — published an article and posted a link to it in the W3C mailing list used by SOAP’s key engineers. While some rigorously developed applications may be thoroughly screened for security holes, MacLeod wrote, the vast majority of applications will never have security as a high priority. He went on to write that the syntax and content of RPCs are based on APIs, and that APIs are subject to frequent change. Every time an API is altered or amended, wrote MacLeod, security analysts would need to reassess the implications.

Many who began dismissing SOAP in the belief that it would be insecure, according to professional developer James Snell, author of a forthcoming SOAP book for O’Reilly, may have done so because it was originally marketed as a great way to do RPC.”

Speculation arose that SOAP could lead to a nightmare situation where one program could automatically hook deep into another program — and the owner would have no idea what had been done, and no way to prevent it.

There’s been a lot of concern it’s not a secure protocol because they didn’t define any security, says Snell. He explained, however, that security was never SOAP’s intention: It’s just an envelope for packaging data. In other words, you can’t blame an envelope for not being a safe. To reassure those who are still worried, Snell adds, Nothing in SOAP is automatic; just by using SOAP, your system doesn’t automatically open up.

Snell’s viewpoints are shared by many at W3C, including representatives of Xerox, who recently posted this:

Authentication, encryption and reliable delivery are already addressed at the level of protocols like HTTP and SMTP.

RPCs and the sessions that bind them are inherently complex, the Xerox engineers wrote, and any attempt by the XML Protocol to address these complexities would be redundant.

The XML Protocol is evolving into a way of using XML to encode data in a way that anybody can read it, no matter what operating system or language, according to Snell.

People should think more about the concept of interoperability than merely RPC. SOAP is more like a universal API. Without SOAP, you’re constantly writing specific APIs between applications — CORBA apps can speak only to CORBA apps, COM apps to COM apps. With SOAP, CORBA can natively interact with COM and vice versa. It’s an Internet standard way of communicating — no single company can get a lock in.

Securing Remote Procedure Calls

As the company best known worldwide for being able to get a lock in, Microsoft is recognized today as SOAP’s leading proponent. Microsoft promotes SOAP as a lightweight protocol for the exchange of both information and RPCs in a decentralized, distributed, networked environment.

The principle of RPCs dates back to Microsoft’s creation of the Component Object Model (COM), a way for small parts of programs (libraries) to be linked together as one program at the time the user runs the application, as opposed to the time the programmer compiles its source code. To move COM out of the confines of a single processor and over the Internet, Microsoft developed Distributed COM (DCOM), which let Windows applications make RPCs to other Windows apps.

Ironically, developers were originally attracted to SOAP, says Dix, because of some of the security nightmares they faced when trying to do DCOM over the Internet. It just was hard to get working, if you could do it at all. The security issues were just awful. CORBA had its own complexities as well. SOAP was written with the intent that people have to work inside of a corporate environment with a firewall, and need to be able to perform the sort of functionality. I know, the IT managers, as soon as you start talking about sending remote procedure calls as HTTP, get a chill down their spine.

SOAP’s dependence on HTTP and Internet port 80 as its primary transfer medium is a method that has been affectionately dubbed tunneling over firewalls. Although this sounds like a built-in measure for security breaches, Dix says, the technique actually relies upon most firewalls’ open acceptance of port 80 to get the message across. Because SOAP is XML and because it is transport independent, he says, it can be — and in the early examples, it has been — applied to sending messages as HTTP over port 80, and thereby circumventing some of the security issues with the firewall. Within the protocol and the specification, however, there are ways of identifying and using HTTP headers, and SOAP headers as well. As Dix explains, targeting SOAP messages for port 80 can enable content filters at the receiving end to scan for explicitly labeled SOAP messages — which could, if an administrator deemed it necessary, be blocked.

Here’s one example of a conceivably common SOAP session: A word processor could use HTTP to place a remote call in XML to a language translation application, requesting that its document be translated into a foreign language. The remote application would respond with an XML document containing the translation, in such a way that the end user would never be aware of the remote application.

How would these applications identify one another, and how is the exchanged data secured? Intentionally, SOAP by itself addresses neither question. Messages between applications are sent in the clear, meaning that anyone who intercepts the transmission and can read basic XML will have access to this information.

To protect yourself, advises Snell, you should at the very least encrypt the memo, as you would with confidential e-mail. In addition, you could send it over SSL, rather than insecure HTTP. Further, you could encrypt the SOAP envelope itself. A method for doing precisely that last item may have just arrived.

XKMS: A Solution In The Works?

The recent announcement by Microsoft, VeriSign and webMethods of a secure XML specification for digital signatures and encryption, called XML Key Management Specification (XKMS), promises to provide some security and peace of mind, at least for users. XKMS is a specification for managing public keys used to support digital signatures or encryption, or other applications of public keys, Verisign’s chief technology officer Warwick Ford tells us. So, it’s designed to work specifically alongside, and in conjunction with, the recent XML signature standard prepared jointly by IETF and W3C.

XKMS offers tools for digitally signing and encrypting documents shared between SOAP applications. So conceivably, a spreadsheet sent between computers could be both protected and authenticated, without engineers needing to amend SOAP itself. Although examples of XKMS for distributed object signing have yet to be investigated, Ford says, XKMS supports signing of XML objects…like business transactions. But it’s not limited to that. So, indeed, if you wanted to build some kind of software distribution system, which was itself an XML application, then you could use this mechanism for signing those objects.

Designing applications properly is the best way to minimize security holes, says Snell. It goes back to good application design — any application is insecure if you use it improperly. If a developer uses SOAP to write an application that accepts application code and then executes that code without first discovering where that code is coming from — ensuring that trusted relationship — that developer should be fired.

Dix suggests that security could be built into a SOAP enabled application by restricting the number of functions it exposes to the outside world — in developer parlance, by limiting its interface. Almost exclusively, says Dix, SOAP applications would not open up [broad] access to the components that exist on the server. Instead, he says, developers should adopt a very focused solution, one that was geared to exposing the functionality of one or a handful of components that you might have on the server.

As late as this week, proposals were being entertained by W3C to drop references to security measures in its upcoming XML Protocol draft, in favor of encouraging applications developers to build security into their own programs, and network administrators to monitor the communications channel. Whatever group provides the final answer to the XML Protocol security dilemma, it is now fair to assume that SOAP’s inner circle of engineers will not be part of it. Developers and security experts may have a rough job ahead.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Securing SOAP. You can follow him on twitter here or receive his weekly newsletter by subscribing here.