Looking Back

In the ensuing 6 years, I’ve been going over and over what went wrong and discovered more lessons along the way: the market conditions were wrong, we were young and arrogant, and, for the most part, we didn’t really understand the magnitude of what we were trying to accomplish: to remind people, our goal was to launch a website in 16 countries (15 EU countries + the US) on day one, localizing our site for each of them. At the time (1999), no one had accomplished that broad a coverage (nor had anyone even tried to).

So it seemed a little crazy but, then again, crazy people had built Netscape, Yahoo, Ebay, and Amazon in the previous few years. So crazy seemed not only possible but it seemed to be the key to success on the Internet. The problems we encountered fell in a number of areas: currency exchanges, tax issues, language localization, integration with many fulfillment partners and a front-end experience that called for broadband connections. We basically wanted to build eCommerce 2.0 long before there was a web 2.0.

Looking Forward

So fast-forward to now. Broadband uptake is nearing 50% in many of the target countries and the number of users has grown tremendously, governments have learned about internet ecommerce and now have specific rules relating to it. And integration across many system is what web services and mash-ups are all about. Do I smell progress? So let’s revisit my old post (which later was published in Business 2.0) points and look at them through the 2006 lens.

The Currency Problem

Back then, the 16 countries we targeted meant 16 different currencies.

Today, with the rise of the Euro as a unifying currency, the same 16 countries only have 4 different currencies (the UK still being stuck on the pound sterling and Denmark keeping its currency a national one pegged to the Euro. The US and the Euro are the other two currencies covered.) This greatly reduces the complexity of pricing models across Europe and makes the overall cost of managing the catalog much lower.

Back then, we actually had to build our own currency tracker, with people inputing the exchange rates daily into the system to keep everything aligned.

Today, you can get access to currency exchanges via web services (just off the top of my head, I can think of Reuters and CBS Marketwatch providing this type of data), therefore automating what was once a manual task and, once again, reducing administration costs for the catalog.

Tax Issues

Back then, there was no consistency in the way taxes were assessed on goods sold online. The financial people at Boo.com version 1 spent a lot of time with a big 5 accountant group and a lot of local government to lobby for normalization of rules around taxes on cross-border business.

Today, because all of those governments understand the value of internet commerce and because many have worked in conjunctions with each other (through the G8 and the EU) to normalize rules surrounding taxation of goods sold on the Internet the problem is easier to solve.

Back then, we had to build our own systems to track all the vagaries of the different tax systems. It wasn’t a build vs. buy decision because there were no packages offered on the market to deal with this.

Today, you can buy software packages that has all the taxation rules built in so that problem is no longer one you need to build for. You can just buy the technology and let the vendor worry about the changes in taxation laws.

Language Localization

When we set out to build Boo.com, a strong component was the idea of offering the online store in the local language of the user. Boo.com was actually the first store to offer as high a level of customization by market and we had to make a number of changes to the e-commerce software package to make it into a globalized platform. Remember that, at the time, e-commerce was primarily the domain of US and UK companies so selling in a language other than English was rare. E-commerce sites which sold goods in non-English markets were generally customized on a one off basis but no one, prior to Boo.com, had attempted to have a single back-end system run multiple countries.

Today, more vendors are selling solutions which can be customized across a variety of western languages. The solutions are not yet perfect but, for the most part, they work (there are still a number of issues when it comes to localization across 2-byte languages, especially when it comes to site with mixed languages.) Back then, we also had to develop a content management system that could handle translation workflows and management of content in multiple languages. It wasn’t pretty but it worked and it required a lot of internal translation to happen. Each product had description, sizes, etc… available in multiple languages. That part was actually a fairly large management of content nightmare. Today, modern content management system can handle more complex workflows (allowing to track when translations are completed) and even can provide hooks to farm-out translation of the content to external parties. This substantially reduces the cost of a multi-country offering.

Integration with fulfillment partners

Back then, a fair number of people at Boo.com were experts in EDI (or electronic data infrastructure) because EDI bridges were the only way to integrate into our fulfillment partners. Web services didn’t exist so we had batch jobs triggering every hour to the warehouses at DeutchePost and UPS so they could pick, pack and ship the orders. This was expensive and probably the area where we lost the most money on a single transaction.

Today, services like fulfillment by Amazon provide the same service at a substantially lower cost and with less integration headaches as web services are making it easy to integrate their services into an e-commerce operation. That saving alone could justify the existence of Boo.com 2.0 (actually, it would be 3.0 as FashionMall tried to resurrect Boo.com once already).

Front-end

No discussion of Boo.com can be full unless we talk about its front-end.

The Broadband Penetration Problem 

Many people laughed at the attempt we made at creating a more user friendly interface to e-commerce. Back then, a more interactive experience meant using Flash. It was the only way to get a lot of parts moving together. Things like Zoom-In/Zoom-out or Rotate type of effects were hard to accomplish with DHTML and much easier to do so with Flash. Since XML didn’t exist, we didn’t have AJAX. Since we didn’t have AJAX, we went with Flash. Since we went with Flash, the assets were large. Since the assets were large and the average user was connecting via a 56k modem, the site looked slow.

The idea was that every click should feel snappy, a model now common with AJAX-based applications but we failed in one assumption, which is that broadband penetration would move at a faster rate. Our expectation were that 1Megabit lines (much slower than what one now gets via cable or DSL) would be readily available within a year. That was a very flawed assumption and we had not planned any contingency for any slower a deployment.

Selling clothes requires details

Another interesting challenge was that we were trying to sell clothes online. Evaluating a DVD, CD, or book online is easy. However, clothing is different: when people shop for clothes, they like to feel the fabric, look at the details in the fabric. That experience was hard to reproduce online. Back then, what we set out to do, in order to help mimic some of the experience was to have highly detailed pictures of the goods. 

Every product was shot multiple times at a stunning 5 megapixels per picture (the highest possible resolution at the time). This meant picture files that were about 1–2 Mb per file, something that seems small in the era of Flickr and YouTube but was massive in the era of 56k modems. The advantage of such detailed pictures was that you could zoom in to a level higher than what you could do in a store (part of our attempt to compensate for the fact that you couldn’t touch the merchandise). Today, such level of detail is standard among most of the online clothing manufacturers and with more broadband lines, it’s no big deal.

Another innovation we introduced was the presentation of products in 3D. You could basically rotate every product in our inventory any way you wanted. This, at a time when QuickTimeVR was not on the marketplace. This meant getting our photography partners to come up with completely new approaches to taking product shots, sometimes requiring as many as 15–20 shots per product in order to get everything right. Those pictures were then taken into Flash and adjusted so that you could rotate the product and zoom in and out of it, a feat that now seems pretty standard, using QuickTimeVR.

All that photography work didn’t come cheap, especially when you consider that this was done across 5,000 products and that all the assets were then stored on our servers (Hard Drive space was nowhere near as cheap as it is now). 

Modeling

Another innovation was the introduction of virtual models you could use to try the clothes on. Today, Sears offers a lower quality version of what we were offering back then (their model still requires a reload of the full page to turn it.) Because all the products had 3D equivalent, modeling them was relatively easy and we decided to throw it in as an extra feature that helped enhance the user experience. Once again, because of the processing and bandwidth required to make that happen, the idea was ahead of its time. 

Miss Boo

So we now all know that chatty avatars on web sites are not a good idea. The concept behind Miss Boo was to help make the experience similar to that of a store, with a sales assistant (Miss Boo), helping you out. Our long term goal was to have Miss Boo attached on the back-end to a real person so we could have integrated IM while you were shopping (that plan never came to fruition as the company had other concerns after launch). In the process, though, we’ve learned that avatars are generally despised and probably helped many sites avoid them.

Tagging

Because we wanted the experience to be a more communal one, we had a way for users to tag clothing (well, we didn’t call them tags, we called them “LaBOOls” (labels, with a Boo in the middle, get it?) in the great tradition of badly named things on our site). However, because there was no AJAX or other way to quickly get the data back and forth, it required a reload of the whole page after each tag was applied. The feature was quickly killed in order to gain speed but I can’t think of any other site that had tagging on products at the time (if I’m wrong, please rectify me in the comments).

Chatty Tone

The BooZine (Boo Magazine) was our attempt to create a more friendly, open tone when dealing with users. We didn’t want to be just a store, we wanted to engage the users. When our forums (remember, this is before blogs were popular) started filling up with vitriolic comments, we were forced to shut them down, closing a channel of communication for users to us. It was a real shame but I think our attempt can be mirrored in the way most web 2.0 companies now have a blog that they use to receive feedback from users.

A more mature market

Back then, few people were buying stuff online. Even fewer were buying clothes online and an even smaller number than that was buying hip clothing. Considering all the challenges Boo.com was trying to address, its target market was just too small to make it a successful business.

Today, blogs like CoolHunting, HypeBeast or MocoLoco show that there is a market for the types of goods Boo was trying to sell. That, in itself, could be a good reason for Boo.com to come back: The market they were addressing is finally there. However, it may also be a reason for it to not comeback: the market they were addressing now has competitors in it.

Was Boo.com the first Web 2.0 company?

I have to admit that I’ve been feeling a certain level of uneasiness about Web 2.0: to me, there didn’t seem to be much there that I had not seen before: web services (yup, done since 2000), user generated content (tried it in a limited fashion with with the “labools” and forums), more transparency (tried that with forums in the past), chatty tone (attempted at Boo). What I failed to realize is that where we failed was in the way we implemented things. But looking back now, the reason it didn’t feel new was that much of that experimentation was on our site only, not part of a more widespread phenomenon.

Another thing that got me thinking along the way of Boo.com as a Web 2.0 company was the excellent post on Pixel Acres about the visual design of web 2.0. Let me explain, picking points from the article:

Integral to Web 2.0 is harnessing the input of website visitors. Users can generate content for a web service, promote it in a “viral” peer-to-peer fashion, and improve it’s data quality through their opinions and preferences.

Users of Boo could create their model, share it with friends (following the UGC model, I guess). So the input component was there, as was the sharing one.

Most Web 2.0 sites come across as friendly, approachable and small-scale, using subtle design decisions to gain our trust.

Every decision about the front end was to make it appear friendly, chatty and hide as much of the complexity as possible (that’s why so many people thought what we were doing was easy but badly implemented).

Bright, cheerful colors dominate Web 2.0 sites… Bold primary colors suggest a playful, fun attitude and also help to draw attention to important page elements.

One word: orange. The boo.com site had cheerful colors all over the place (sometimes so cheerful that I worried it would be seen as a toy)

Rounded Everything: The friendliness of rounded corners is in keeping with the comfortable, informal tone of many web 2.0 sites… In a great FontShop article analysing the logos of Web 2.0, it was clear that rounded typefaces are all the rage. This smooth approach to type lends a modern playfulness to a company’s visual identity.

Yup, Boo.com was round, very round, even the logo and the fonts. From a visual standpoint, it was much closer to today’s web 2.0 site than the ones it lived among.

Most Web 2.0 sites devote prime real estate to the message that they offer a free service.

Well, we kept pushing our “Free” boozine (Boo Magazine) and looked at it as a way to hook people into coming back again and again to the site.

You won’t find any stock photography of smiling support staff on a Web 2.0 site — that’s a tactic favored by small companies trying to mimic large corporations. Simple icons and screenshots are the order of the day when it comes to imagery on Web 2.0 sites. 3D and beveled icons can lend elegance and polish to a page design that is otherwise fairly stark.

Boo.com was 100% stock photography free. It was all icons and cartoons.

A good Web 2.0 app ought to be lightweight and easy for users to grasp, and clever visual design and copywriting can help remove barriers to entry. Smart use of layout, color, type and copy can go a long way towards easing the pain.

Well, we failed on the lightweight end of things but the design was to be as airy as possible.

As far as Web 2.0 is concerned, bigger is definitely better. Bigger text, that is. Large text is easy on the eye, and coupled with snappy copywriting makes information easy to absorb. And now that accessibility is cool, it’s possible to be a hotshot web designer and use enormous type.

… and back then, people said we didn’t make good use of the real estate because the fonts on our screens were too big. However, note that accessibility was inexistant at Boo.com

The layout of Web 2.0 sites might be described as minimal. With a focus on legibility and ease of use, good use is made of white space. White space allows important information to stand apart, provides rest for the eye, and imparts a sense of calm and order. Generous leading also makes text copy easier for the eye to follow. Some Web 2.0 layouts are so minimal that they verge on boring, but designed well, an uncluttered page can be incredibly tasteful.

Yes, we had a lot of whitespace.

Friendly, informal copywriting allows a more personal relationship with website visitors.

People complained that our content was too informal, actually. I guess taste has changed in the following years.

So, from a visual standpoint, we may have established some of the rules that are now considered good visual rules for Web 2.0 companies. Of course, feature wise, we didn’t have RSS (it had not achieved the level of popularity it now has) and worked largely as a walled garden (all interaction happened on our site) but Boo.com was probably sitting closer to a Web 2.0 sensibility than most companies that existed at the time.

Conclusion

Based on past history, the complexity that existed back then has largely disappeared, making it possible for Boo.com to exist in the web 2.0 world. The market has also evolved to the point where many of the innovations first introduced by Boo.com are now considered mainstream and where many of its barriers to entry seem to have disappeared. This means that Boo.com could have a chance at surviving this round. However, one would have to be careful about overspending on advertising (a crime that Boo.com was responsible of, with its massive multi-country ad budget). A question that remains on the viability of the brand is whether the errors of the past have damaged the brand to a point where it would not be able to come back. It is probably the most dangerous factor in the rebirth of Boo.com and, if the negative press of the past overshadows the re-emergence of this company, it could be a fatal flaw that could ultimately make this a bad idea.

I wish much luck to the parties involved in the relaunch. Hopefully, they won’t suffer from the same arrogance we suffered from in the first iteration of the company and will be able to build a strong business around this brand.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Why the Boo.comeback makes sense. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

• Yahoo! announcing native support for RSS and ATOM in the “my yahoo” page
• Bloglines announcing a new set of services to ease the load of distribution
• Newsgator announcing new partnerships to drive adoption

Let’s review why those announcements herald the arrival of RSS into the mainstream.

Yahoo! and RSS

As Jeremy Zawodny said,

it does something […]–something that Yahoo is in a unique position to do: bring RSS to the masses.

Why is this significant? Well, quite simply, while geeks like myself and readers of this blog know, RSS is still something for early adopters. Every time a large player gets into that field, the concept gains a little more traction. With the arrival of RSS into the Yahoo! personal page, the format becomes a major new delivery channel for content creators. With this, TNL.net can now figures prominently next to Reuters feed, being given the same kind of weight.

It represents a major shift in the way Yahoo! distributes content. In the late 1990s, the only way one could get onto that page was to strike a relationship with Yahoo! This meant spending times with both your and yahoo’s lawyers and resulted in fewer spots being available on that personal page (For the sake of disclosure, I was involved in one of those deals, providing content from Internet.com on the Yahoo! pages). Looking at this situation, Netscape developed a new format to push content into their personal page. That format was the genesis for what we now know as RSS. So Yahoo! has now taken a strategy that was initially developed by Netscape as a way to compete with Yahoo! Oh, the irony!

With Yahoo! making this move, one can only wonder as to when other portal players will do. Will they follow suit? Will AOL soon support RSS? What about MSN? If that’s the case, this would be a seismic change in the level of support for RSS and would get most major companies to start thinking about distributing content over this channel.

Bloglines, Newsgator and distribution

More geeky but also of high significance are the announcements that Bloglines is offering web services which can be integrated in a number of other newsreaders, and a similar announcement from Newsgator. A few weeks ago, I wrote about capacity planning and RSS. This is a major issue in the world of syndication these days and Bloglines is the first service to attempt to offer an open solution. Similarly, Newsgator’s announcement represents the fact that such approach is something that more people are thinking of, not just the works of a single company. The reason these technical announcements are particularly significant is that they represent a potential shift in terms of how people think of RSS.

The distribution to a super POP for RSS can represent a step in the right direction in terms of distributing syndication feeds. One could envision feeds being distributed to entities like Bloglines as a way to alleviate and redistribute traffic. With more and more messaging moving over RSS, this could be an interesting approach in terms of making RSS scale.

The fact that Newsgator also takes that approach shows that this is something that is gaining support. I do see the Bloglines effort as having more of an impact, however, because it is an open one, whereas Newsgator is creating more of a walled garden by partnering strategically instead of offering hooks into the service to anyone who wants to use it.

What we are seeing here is the emergence of a new kind of content aggregators and distributors. They are the new pipeline, the new glue for what could become a new way to distribute content. And I don’t just mean the kind of light text-based content which currently makes up the bulk of syndication feeds. What I can envision is these kind of efforts becoming similar to bittorrent, offering audio, video, and other types of rich content in a distributed fashion. At the end of the day, they could become potential competitors with Akamai, as they push more content closer to the edge without overloading the servers offering that content. They can potentially become the new proxies of syndication.

However, this can also become a point of concern. As more and more people rely on such points of presence, they have to created a trusted relationships with the providers. What happens if the service goes down? What if they decide that a feed shouldn’t be redistributed ? What happens if they decided to add advertising to every item in a feed? (I’m not saying that they would think in such a way but trying to highlight a potential scenario)? What would happen if a legal entity asked them to stop distributing a feed? There are a number of interesting issues arising out of the concept of redistribution. While in the past, Bloglines could only affect blogline readers (and Newsgator could only affect Newsgator users), they now sit at a major intersection where they become a critical part of the architecture.

This represents issues but also opportunities. While they do aggregate content, alleviating load from the server of content creators, they also sit at a point where they can get more information about readers. For example, a simple question thrown on one of those services could allow to create better demographic profiles of syndicated feed users. One could see them aggregate that data and start reselling it to content creators who are interested in knowing more about theirs readers. In the future, one could also start relying on those networks as a way to target readers (for example, target particular entries to particular types). And finally, those networks could finally offer the holy grail of syndication: monetization. Since they would be sitting in a space where they can control a walled garden, they could start offering “premium” services, subscription to feeds (or baskets of feeds) for a price, going into a revenue sharing model with the content creators.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title RSS on Fire. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

With the new agreement, AOL has received a royalty-free license to use Internet Explorer for the next seven years. Since the browser has been sitting at the core of their online service client, it is doubtful that this will change in the future. As a result, AOL is now supporting an open source project which adds little value to its bottom line. The Netscape browser holds very little strategic value for the company moving forward. Considering its enormous debt, AOL Time-Warner might eventually reconsider its investment in the Mozilla project.

In its initial iteration, a large part of the development for Mozilla was done by Netscape developers. In fact, the Mozilla browser is distributed under the Netscape Public License, which still ensures that the company has some level of control over what goes on there. While it is an open source, it is one with a clear sponsor.

And that sponsor may now rethink its participation. So who will pick up the slack once they do?

My best bet on this is that IBM will step in if this happens. While it may seem like an odd choice, it seems to be the only logical one when studying the matter more closely. First of all, the company has been making sizable investments in Linux has already paid some hefty dividend for the company and has allowed it to gain entry into new markets. As a result, IBM is placing itself as a clear competitor to Microsoft on the mid-range to high-range server end, using open source projects as its own horse in that race.

But why should it matter to IBM, one might wonder? Well, for starters, the server market is the entry point to larger scale application offerings in the future. With the era of web services now upon us, IBM wants to make sure that it will still remain relevant moving forward. The web services world is one in which both IBM and Microsoft are currently happy to play together, jointly defining specifications for the space, there is a clear understanding that they are in competition for this future space. History has obviously shown that Microsoft and IBM can partner up and have eventual fallouts. It is a lesson that probably was not lost on IBM and that now permeates a lot of what they do.

So back to Mozilla. While it is mostly seen as a browser, Mozilla is much more than that. For starters, it offers a complete suite of Internet products, ranging from the well known browser to a bug tracking system, an email client and much much more. At its core, Mozilla is a development platform on which other applications can run. This is significant in that it provides a cross-platform development environment which offers a nice alternative to the Microsoft windows platform. So, while Microsoft is trying to use the net as a way to bring everything back into windows, Mozilla can be used to bring net components into a variety of operating systems (hence benefiting Linux, because you are not locked into a particular operating system).

As more and more application require access to the Internet, the browser window becomes the de-facto UI for the computing world. It’s something that Microsoft understands, and this is why they are more tightly integrating the browser into the OS and why tools like Mozilla are important.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Mozilla after AOL. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

If the trend holds, we will increasingly see extremely specific application modules being developed instead of one-size-fits-all software. That, in turn, might erode the profit margins of software development companies as they will be unable to sell features that the customer does not want.

As this more distributed model evolves and services become less and less dependent on the underlying operating system, what will happen to companies like Microsoft, who tie things very closely with their operating system?

It seems to me that the software world is about the experience the kind of breaking point the music industry has experienced with the rise of Napster. Napster was significant not only because it allowed to share music but because it unbundled songs, making it difficult to sell a whole album solely because there was one of two good songs on it. If software features become decoupled due to web services, we could see software companies being forced to lower the price of their software packages and sell features on a new pricing model. Large companies like Microsoft will not be killed by large competitors but by the emergence of thousands of small competitors who will build better features.

An interesting trend that should be followed.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Death by a thousand papercuts. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

A Rapid Epidemic

Now that I’ve got your attention, let’s take a quick look at how Code Red spread. First of all, there was a simple buffer overflow problem in Microsoft Index Server, for which the company produced a patch. A month later, Code Red starting showing up. However, its rate of growth was relatively slow at the beginning. The true epidemic did not start until July 19th, when Code Red exploded onto the scene, increasing the number of infected servers from just around 300 at 00:15am to 2994 by 7:30am, over 30,000 by 14:40pm and over 300,000 in the 6 hours after that. In other words, in less than a day, Code Red went from a relatively small annoyance to a full blown attack on the net infrastructure. Had no one rung the bell on it, it would have taken only a couple of days for it to infest every single version of Microsoft IIS (or about a quarter of all web sites on the net).

Who’s responsible?

While the hunt is on for the person who devised this virus, the list of people who have some level of responsibility in the spread of this virus is a very scary one: Microsoft, of course, for first putting out a faulty product, a web server with a big security hole. However, credit goes to Microsoft for putting out a patch before they even knew of the worms’ existence.

Another group which deserves some blame is IIS system administrator of infected systems. Let’s face it, we all know that Microsoft software is riddled with holes. We also know that Microsoft puts out patches on a regular basis. We all know that those patches solve most of the problems before they occur. Well, the people who were infected by Code Red did not follow the basic rule of patching systems early and often.

Many Unix administrators are laughing right now at IIS users: they shouldn’t!

The security on some Linux systems is so dismal that a virus similar to code red but aimed at Apache servers could have done much more damage much more quickly.

What have we learned?

The first thing that we have learned out of the code red offensive is that we can’t rely on people to update their systems properly. When system administrators fail at that task, they contribute to a lower level of security for the net as a whole. But SAs are human and to err is human. The truly scary thing is that Code Red is only the first of a series of viruses that will gain preeminence on the net.

The truly scary thing is when an application used by general consumers gets attacked by a similar worm.

Earlier this year, I covered a set of security problems in AOL’s Instant Messenger. One of the ways hackers are taking over AIM is using buffer overflow, throwing large strings of apparently nonsensical characters to the client in order to take it over. Unfortunately, Code Red acted the same way with IIS servers. What will happen when someone uses the Code Red approach to create an attack on AIM? The thought sends chills down my spine as I can see hundreds of millions of computers acting as zombies in a possible net-wide denial of service attack.

Mutations

During the August 1st attack, webmasters noticed not one but two (and possibly three) different types of code red attacks. The first one was the same as the July 20th worm but the second was much more nefarious, a worm that did not announce itself (it did not deface web sites) but instead added a backdoor allowing hackers access to the compromised servers. When work done on one virus reappears in another, we call it a mutation. What is truly worrisome is that Code Red was not a very sophisticated virus. Others, like Hybris can update themselves.

What happens when Code Red is merged with one of those other viruses? This is yet another scary question that I send out for discussion.

Infrastructure

For the past year, I’ve been paying more attention to the security space. I wasn’t sure of why but I felt that this was an area I needed to pay attention to. More and more of our technological infrastructures are moving to the Internet. These days, telephone companies, cable companies and many others are tying into the grid. However, it seems to me that little attention is being given to security. As we move into the era of web services, there needs to be an important dialogue in our industry as to how we increase the security and reliability of the Internet.

Furthermore, as new operating systems come out, they should be thoroughly proofed for security holes. Apple recently released OSX and already, a number of holes are being noticed. Microsoft is still set on releasing Windows XP within the next few months but few in the security community have had a chance to test out its security. With all that said and done, I would also like to encourage all of you to question whether any of your connections are secure. For example, if you are running a DSL or cable line at home, have you firewalled your environment? These days, it’s little things like that that make a lot of difference.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Seeing Red. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the meantime — and possibly as a result– Microsoft and Verisign have just announced a new security procedure for person-to-person SOAP transactions, but a workable mechanism for securing Internet transactions between software and software may be years away.

Some of SOAP’s architects contend that building security into their protocol would only sacrifice its simplicity, and that the HTTP sessions that SOAP transactions rely on can already be secured at the session level, with protocols such as SSL. Moreover, securing sessions from outside interception, security experts believe, cannot protect transactions from two other perceived threats: interception from the inside and bad programming. With a protocol extension to SOAP for message attachments in the works, a third possible threat emerges — one that too many have become familiar with: malicious scripts.

Chris Dix, a SOAP programmer with FMStrategies, sides with the majority in believing that it may now be incumbent upon developers to endow applications with the specific security measures they need to communicate on open networks: If you opened up your [program’s communication] interface to be broad enough to accept things that might be dangerous, Dix says, then it would be your responsibility as a developer to make sure that the requests that might be dangerous came from people who knew what they were doing, and that you built in security.

Unlike the exchange of documents — spreadsheets and word processor files — between two people who can use public key infrastructure (PKI) or other measures to identify each other, distributed software components will communicate with one another without human intervention. In the new net services platforms such as Microsoft’s .Net, Novell’s One Net and Genuity’s Black Rocket, distributed software components will be everywhere, placing remote procedure calls (RPCs) to one another using the XML protocol. So why is the W3C close to deciding that security is not an issue, at least for them?

Should W3C Address Security Concerns?

The security debate began last May, when Ken MacLeod, an engineer of XML-RPC — a SOAP forerunner — published an article and posted a link to it in the W3C mailing list used by SOAP’s key engineers. While some rigorously developed applications may be thoroughly screened for security holes, MacLeod wrote, the vast majority of applications will never have security as a high priority. He went on to write that the syntax and content of RPCs are based on APIs, and that APIs are subject to frequent change. Every time an API is altered or amended, wrote MacLeod, security analysts would need to reassess the implications.

Many who began dismissing SOAP in the belief that it would be insecure, according to professional developer James Snell, author of a forthcoming SOAP book for O’Reilly, may have done so because it was originally marketed as a great way to do RPC.”

Speculation arose that SOAP could lead to a nightmare situation where one program could automatically hook deep into another program — and the owner would have no idea what had been done, and no way to prevent it.

There’s been a lot of concern it’s not a secure protocol because they didn’t define any security, says Snell. He explained, however, that security was never SOAP’s intention: It’s just an envelope for packaging data. In other words, you can’t blame an envelope for not being a safe. To reassure those who are still worried, Snell adds, Nothing in SOAP is automatic; just by using SOAP, your system doesn’t automatically open up.

Snell’s viewpoints are shared by many at W3C, including representatives of Xerox, who recently posted this:

Authentication, encryption and reliable delivery are already addressed at the level of protocols like HTTP and SMTP.

RPCs and the sessions that bind them are inherently complex, the Xerox engineers wrote, and any attempt by the XML Protocol to address these complexities would be redundant.

The XML Protocol is evolving into a way of using XML to encode data in a way that anybody can read it, no matter what operating system or language, according to Snell.

People should think more about the concept of interoperability than merely RPC. SOAP is more like a universal API. Without SOAP, you’re constantly writing specific APIs between applications — CORBA apps can speak only to CORBA apps, COM apps to COM apps. With SOAP, CORBA can natively interact with COM and vice versa. It’s an Internet standard way of communicating — no single company can get a lock in.

Securing Remote Procedure Calls

As the company best known worldwide for being able to get a lock in, Microsoft is recognized today as SOAP’s leading proponent. Microsoft promotes SOAP as a lightweight protocol for the exchange of both information and RPCs in a decentralized, distributed, networked environment.

The principle of RPCs dates back to Microsoft’s creation of the Component Object Model (COM), a way for small parts of programs (libraries) to be linked together as one program at the time the user runs the application, as opposed to the time the programmer compiles its source code. To move COM out of the confines of a single processor and over the Internet, Microsoft developed Distributed COM (DCOM), which let Windows applications make RPCs to other Windows apps.

Ironically, developers were originally attracted to SOAP, says Dix, because of some of the security nightmares they faced when trying to do DCOM over the Internet. It just was hard to get working, if you could do it at all. The security issues were just awful. CORBA had its own complexities as well. SOAP was written with the intent that people have to work inside of a corporate environment with a firewall, and need to be able to perform the sort of functionality. I know, the IT managers, as soon as you start talking about sending remote procedure calls as HTTP, get a chill down their spine.

SOAP’s dependence on HTTP and Internet port 80 as its primary transfer medium is a method that has been affectionately dubbed tunneling over firewalls. Although this sounds like a built-in measure for security breaches, Dix says, the technique actually relies upon most firewalls’ open acceptance of port 80 to get the message across. Because SOAP is XML and because it is transport independent, he says, it can be — and in the early examples, it has been — applied to sending messages as HTTP over port 80, and thereby circumventing some of the security issues with the firewall. Within the protocol and the specification, however, there are ways of identifying and using HTTP headers, and SOAP headers as well. As Dix explains, targeting SOAP messages for port 80 can enable content filters at the receiving end to scan for explicitly labeled SOAP messages — which could, if an administrator deemed it necessary, be blocked.

Here’s one example of a conceivably common SOAP session: A word processor could use HTTP to place a remote call in XML to a language translation application, requesting that its document be translated into a foreign language. The remote application would respond with an XML document containing the translation, in such a way that the end user would never be aware of the remote application.

How would these applications identify one another, and how is the exchanged data secured? Intentionally, SOAP by itself addresses neither question. Messages between applications are sent in the clear, meaning that anyone who intercepts the transmission and can read basic XML will have access to this information.

To protect yourself, advises Snell, you should at the very least encrypt the memo, as you would with confidential e-mail. In addition, you could send it over SSL, rather than insecure HTTP. Further, you could encrypt the SOAP envelope itself. A method for doing precisely that last item may have just arrived.

XKMS: A Solution In The Works?

The recent announcement by Microsoft, VeriSign and webMethods of a secure XML specification for digital signatures and encryption, called XML Key Management Specification (XKMS), promises to provide some security and peace of mind, at least for users. XKMS is a specification for managing public keys used to support digital signatures or encryption, or other applications of public keys, Verisign’s chief technology officer Warwick Ford tells us. So, it’s designed to work specifically alongside, and in conjunction with, the recent XML signature standard prepared jointly by IETF and W3C.

XKMS offers tools for digitally signing and encrypting documents shared between SOAP applications. So conceivably, a spreadsheet sent between computers could be both protected and authenticated, without engineers needing to amend SOAP itself. Although examples of XKMS for distributed object signing have yet to be investigated, Ford says, XKMS supports signing of XML objects…like business transactions. But it’s not limited to that. So, indeed, if you wanted to build some kind of software distribution system, which was itself an XML application, then you could use this mechanism for signing those objects.

Designing applications properly is the best way to minimize security holes, says Snell. It goes back to good application design — any application is insecure if you use it improperly. If a developer uses SOAP to write an application that accepts application code and then executes that code without first discovering where that code is coming from — ensuring that trusted relationship — that developer should be fired.

Dix suggests that security could be built into a SOAP enabled application by restricting the number of functions it exposes to the outside world — in developer parlance, by limiting its interface. Almost exclusively, says Dix, SOAP applications would not open up [broad] access to the components that exist on the server. Instead, he says, developers should adopt a very focused solution, one that was geared to exposing the functionality of one or a handful of components that you might have on the server.

As late as this week, proposals were being entertained by W3C to drop references to security measures in its upcoming XML Protocol draft, in favor of encouraging applications developers to build security into their own programs, and network administrators to monitor the communications channel. Whatever group provides the final answer to the XML Protocol security dilemma, it is now fair to assume that SOAP’s inner circle of engineers will not be part of it. Developers and security experts may have a rough job ahead.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Securing SOAP. You can follow him on twitter here or receive his weekly newsletter by subscribing here.