Electronic Consumer Hardware

If you have a stereo or a set of speakers in your house, you’re dealing with technology that hasn’t really changed that much in almost a century. In the same way, TV sets have seen little evolution in the way of being smart over the last 50 years: when you think about it, the biggest steps in TV have moved from black & while to color (in the 70s), cathode ray tube to plasma and LCD (late 90s), and analog to digital (early 2000s). In each case, the focus has been on the picture and not so much on the logic to receive and display information.

While new models of television of radio are introduced on a yearly basis, the fundamentals behind them are essentially the same and the features that are introduced are, for the most part, marginal improvements.

However, over the last 4–5 years, a quiet revolution has been underway in the hardware business with the addition of two apparently innocuous components making their way through into a lot of the more recent version of new devices: USB drives and internet connectivity (either wired or wireless). This has made it possible for the core software components of those devices to be upgraded, either by downloading the upgrade on a USB drive and plugging it into the consumer device or automatically in the background via the internet, opening a whole new world of possibilities.

The remote experience

A radio station in the 1920s used a knob to find the right signal and today it’s a button. The clarity improved by moving from AM to FM and HD radio but the user experience is basically the same. For TV, the experience has moved from having to turn a knob on the TV screen to turning a knob on a device that was connected via a cable to the TV to pressing a button on a wireless remote control. The interface has not changed much, except for adding more buttons to allow access to more complexity.

Meanwhile, an increasing amount of touchscreen devices are starting to populate home, whether they are mobile phones or tablets and there is now a slew of software that allows for operating some of the more complex technologies out there. Using simple infrared adaptors, those devices are now allowed to share their own smarts with TV sets and radios, pushing the creation of simpler and more adaptable interfaces for remote controls to increasingly becoming software based solutions. As software evolves, it can be upgraded easily and include new functionality without getting rid of the devices that consume it.

Hardware is a platform

In this instance, we are seeing software trumping hardware. Short of the dependence on certain physical components being natively included in hardware devices (for example, the ability to support certain forms of communications like infrared or short range radio), the hardware does not really need to get upgraded unless its primary function (eg. showing a picture or playing some sounds) is itself in need of an upgrade. And one could see a time when the receiving hardware would receive software upgrades that allow for this interoperability to be a smoother experience because hardware is a platform.

A group of technologies around interoperability have made it easier for this phenomenon to happen. While few people care about such standards at HTML5, HTTP, TCP/IP, WiFi, and DLNA, they can serve as the building block of the future.

Imagine a television, for example, where every channel would be served through an internal web server that rendered everything on the screen via a web page with an HTML5 embedded video player. All of a sudden, the TV screen would become a giant web browser in full screen mode, allowing to not only access any content on cable or broadcast TV but also any content available on the internet.

Add a layer that would allow for throwing HTML5 applets on top of that screen and you would have a standard compliant approach to developing things for television. Throw a webGL interpreter in there and you have something that is drastically more advanced than what any TV in 2011 can currently do. Now make this layer addressable as a web service standard and you could not only see other companies incorporate it but also see an explosion of support from the development community.

Standardization is they key to TV’s future

Today, developing for television (or send streaming audio to a stereo) is an often frustrating experience, forcing developers to encode content so that it meets the requirement that each TV or set-top box manufacturer has set forward. This often complicated landscape has left most developers avoiding it because the return on investing in a single platform just isn’t there. A way to battle this is through standardization of the underlying interfaces. Since the 1990s many people, myself included, have pushed for a standard language to merge television and the web.

These efforts need the support of TV manufacturers and the understanding from said manufacturers that open standards will not only lift their industry but potentially fuel another area of growth for their offerings. As more and more applications get developed on top of a standard compliant deck, there will be increasing demands on the part of the developers to give access to other parts of the hardware, which could be completed via paid software updates. Hardware manufacturers would then find themselves in a world where they could make money on the initial hardware they sell but also add extra revenue by turning on extra functions through software sales.

Lack of standardization creates a winner-takes-all market

Apple has an early lead in the audio market, with Airplay (and its support from many hardware vendors). For example, in our house, we have equipped several rooms with Airport Express adaptors connected to powered speakers or stereos. Apple had initially locked Airplay to Apple only devices, but an ingenious company called Rogue Amoeba put out a piece of software called Airfoil that allows us to play from any source we can find on a computer. It would be nice to get such functionality on our mobile phones and tablets but, because Apple is locking up its system, it is unlikely that such thing could happen unless allowed by Apple.

An open standard that reproduces such functionality could not only hamper Apple’s ability to lock people into its ecosystem but could also help proliferate the rise of inexpensive devices that can be addressed from the internet.

In this case, as in the case of TV, we could see a single closed proprietary standard emerge and create an early lock-in for the owner of that standard. However, if there is one lesson the internet has taught everyone in the computing industry, it is that open always trumps proprietary in the long run. Early platforms may get early wins, but eventually, the open standard disturbs their marketplace and destroys the proprietary aspect.

Some people may consider the the receiving devices for new content as dumb devices but I would argue that they would fall in the category of smart devices: after all, isn’t the smartest person in a room generally the one that first listens and only speaks their mind after receiving appropriate input. Maybe we should create a new category, calling those wise devices instead of smart ones.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Interop: the future of hardware. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Looking at the history

In order to look forward, it always pays to look back. The dominant standards for the web today are undeniably HTML (or its variances like XHTML) and HTTP. More recently, XML has emerged and, increasingly, RSS is becoming the dominant type of XML for sharing a variety of data.

How did each of those standards become a standard. It is obvious now (hindsight is always 20/20) that standards bodies have relatively little bearing when it comes to influencing the succes of a format. Take, for example, SGML, which was the dominant standardized format for document formatting. It was quickly superceded by HTML which, at the time, was not considered a standard.

The same is true of RSS and other standards for syndication. Formats like ICE, CDF, and NewsML were touted as the future when they were first introduced. However, they’ve recently been superceded by RSS.

And even within the RSS world, formats like RSS 1.0, which was supposed to be more semantically sound, and ATOM, which was supposed to be more forward thinking that RSS 2.0, have been losing the war to RSS 2.0.

Bootstrapping is a social phenomenon

What Dave Winer understood, when he sheperded RSS 2.0 into becoming the dominant mean of delivering syndicated content is that the life and death of a new format is predicated on its widespread adoption. And, in order to increase adoption, one has to make something generic, easy to understand, and simple.

Many of the people in the early days of the syndication space failed to see it as Dave did. We believed that a semantically sound format was better and we were wrong. Purity, it turns out is not always a good thing, especially if it gets in the way of people implementing something.

The same is true of HTML. I’d venture that, from a development standpoint, the biggest boost to HTML was a single menu feature that appeared in early browsers and remains there to this day: view source. In the early days of the web, countless developers learned how to do cool things with HTML by reading the source of pages designed by other people.

In a recent issue of ambidextrous magazine, Jeffrey Schox talks about the three stages of technological development: appropriation, early innovation, and sustainable innovation. Here’s how he describes the appropriation stage:

an issued pattent allows innovators to construct roadblocks behind them as they travel down a particular technological path… During the appropriation stage, patent roadblocks waste time and money… The countries, needing to catch up with the designs and technologies of other countries, should focus on collecting revenue and knowledge streams to fuel later stages of technology development.

While he focuses on hardware and electronics in a globalized marketplace, the same truth can be applied to standards. With few barriers in adopting a new standard and by fostering a culture of appropriation, one can easily establish a base of people who understand a new format. As more people understand it, they start implementing it and, after eventually getting smarter about it, start building on the efforts of previous creators. Eventually, those masses of tinkerers get to a critical point, pushing the new format into areas that were unexpected. Some companies eventually get smart to it and see growth in that area, which triggers them into experimenting with that new format.

Eventually, due to a general agreement among all developers, the format becomes a de facto standard. It does not have to have the imprimatur of a standard body (except for some very late adopters or pockets where such imprint is considered important) and moves forward.

What is interesting is the next stage, the one where standard bodies see the area as hot and decide that they need to play in that field. A good example of that is the ATOM format, which has been enshrined into an IETF approved format, and to date has failed to stop the RSS 2.0 juggernaut.

So what happened?

The amazing thing is how simple the issue is. The reason RSS 2.0 has been winning is that it has developed a following. With every new developer learning RSS 2.0, the format goes stronger and the same is true of every company implementing it. Because it is simple, it’s easy to pick up, which means that new developers can do interesting things with it relatively quickly, giving them a chance to become active members of the community and therefore become hooked on it.

The other issue is in keeping things relatively open, while still maintaining some level of control over the general direction. A successful future standard has to allow people a chance to contribute but, in the end, it also needs some gatekeepers who decide what goes in and what doesn’t. The same truth can be applied to any sofware development cycle: for example, Linux may be a widespread open source phenomenon but the number of people who decide what goes into the core kernel or doesn’t is still relatively limited. The same is true of any successful open source project: some level of centralized decision making and distribution of the work: anyone can contribute but not every contribution makes it into the final product.

I’m now seeing some of the same history repeat itself in the OPML space. It’s a format that is very simple and Dave is working very hard on getting people left and right to support it. It’s the same scenario he’s used to bootstrap the RSS format and to bootstrap concepts like blogging and podcasting into the mainstream. It’s a formula that works: keep it simple to implement, maintain some level of centralized control over the roadmap and then evangelize it left and right until it can no longer be stopped.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Standards as social contracts. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In his note, he says that RSS is broken. I personally believe that at issue is not whether RSS is working or not. RSS is working but it has complicated the bandwidth issue. At issue is the fact that RSS feeds are generally generating more traffic to a site. Because RSS readers are polling the site to check if a feed has been updated, the traffic patterns change, with increased numbers of spikes on a hourly basis. This is similar to some of the issues network administrators started facing when Pointcast first appeared.

There are a number of ways to mitigate the issue.

HTTP Conditional GET for RSS

First of all, one of the things to consider when using RSS is to create conditional HTTP headers on RSS feeds. This helps mitigate some of the impact by ensuring that feeds are only served if the content has changed.

Feed Compression

The next item to think of is to use compression when serving feeds. By doing so, one reduces the size of the payload, which ends up being much better in terms of managing bandwidth. In my own experience, because RSS is primarily text, I’ve seen a reduction of 80% of the bandwidth when delivering RSS feeds in a compressed format. That represents a fairly large gain in bandwidth that can then accommodate more users.

Change the polling schedule

The RSS 2.0 specification already offers a number of optional elements to give RSS readers a better idea as to when to get content. For example, the pubDate element offers information as to when a feed was last published, as does the lastBuildDate one. ttl (aka. time to live) can also be used to indicate to the software that this feed should live for a certain amount of time. Finally, skipHours and skipDays offers more pointers as to when RSS reader software should not poll. With all those mechanisms in place, it looks like a lot of flexibility exists in the format to accommodate scalability.

When all else fails, reduce

If all of the above still fail, RSS publishers should look at reducing the size of their feeds. There are two ways you can do this. First, you can just say that you’re not going to offer full-text feeds. This seems to be the option that Scoble hates. Another way to do things is to offer both abbreviated feeds and full-text feeds or offer more detailed feeds, as I do on TNL.net.

An important consideration when doing something like this is how to address them. By default, users who just use the RSS autodiscovery feature will only get the abbreviated feed. However, they still have the option to go and get the full-text version. The compromise here is that users who just want to subscribe quickly can do so at a lower bandwidth costs, while power users can seek out the fuller feed and subscribe to that. The result, in my experience, is that most people use the autodiscovery feature, grabbing the smaller feed. Some power users do seek out the fuller feed and subscribe to that instead (based on the numbers, I’m seeing a 5% usage of the full-text feed as opposed to the default abbreviated one. This is a compromise solution that seems to accommodate everyone involved to date.

Final considerations

When publishing RSS feeds, your audience grows, which results in traffic growth too. One of the thing to realize is that RSS feeds are generally stickier than the rest of a site. What this means is that, for every new subscriber you get, you will see an on-going increase in your overall site traffic stats. This is not a bad thing as messages emanating from your site do get a higher passive readership. One of the thing that new syndication standards should consider is a follow-up on this. While RSS publisher know how many feeds are being pushed out, there is little, in the way of information as to what percentage of those feeds is being read. Stronger metrics need to be developed to get an understanding of passive vs. active subscribers (passive subscribers are subscribers that receive the feed but do not read it, while active subscribers are actually reading the content and clicking through). This, I believe, is one of the next challenges that needs to be addressed in order to make RSS a more viable and widespread distribution platform.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Capacity planning and RSS. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Here’s why I chose this particular name as my entry:

• First of all it is short. By comparison, people talk about the “web” instead of the “world wide web” so any nickname needs to just roll off the tongue.
• Second, it has to represent the concept: When I get a tip from an RSS feed, what am I getting? First, I’m getting a reminder that something new has been posted. In some cases, it’s a portion of an entry, in others, it’s the whole entry. In either case, I am alerted to the fact that new content has been posted. This alert comes electronically, hence the replacement of a single letter to form the new elert term. An elert is an electronic alert.
• Why no web mention? Well, I thought of that one for a while, as the term web has been more elastic of late but I went back to the early days and the web is only one component of the Internet. Sure, the RSS feeds are generally distributed over HTTP but does that make them part of the web? I would venture that no. Much like email can be gotten via webmail, or Usenet can be reached via a web interface, they represent a different set of applications. The same is true of RSS (or Elerts) as those feeds can be read through a web-based interface (like Bloglines, my favorite RSS reader), or through an individual client. However, no browser natively supports RSS yet (I believe it’s only a question of time, though).

The concept of the contest was great and I hope that everyone will join in (and that most people will support Elert as the coolest entry)

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title The case for Elert. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

There is really nothing I can say that really expresses how amazing I find this move.

By putting the specification under an Attribution/Share Alike Creative Commons license, Dave has essentially manage to make the rights on this specification very clear. This should put an end to any discussion related to how much control Userland exerts on the specification.

The creation of an advisory board is also a move in the right direction. If there is one thing I would recommend, however, it is the election of two extra members that would be voted on by the community. This might be to consider in the future but my congratulations go to all members of the current advisory board. It is a tough job but one that I’m sure you will all do well.

Once again, Dave, thanks for this great gift to the community. This, to me, has to be the most important announcement in the syndication space in several years and will ensure that the specification will move forward, hopefully putting behind us a lot of the personal attacks that have mired the standard recently. 10–20-30 years from now, people will look back and see this moment as equivalent to the first time Tim Berners-Lee typed “http://” and the first time a ping packet was sent.

This is the greatest gift made to the online community since Tim Berners-Lee decided that he was going to give every components of the web to all of us.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Extending the Olive Branch. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

For the past few years, Microsoft has been trying to figure out how to remain relevant in an era of increasing openness. The rise of HTML and of HTTP as the underlying protocol for distribution on the Internet have challenged the level of control that Microsoft had on the computing world. The initial control was borne out of a partnership between Intel and Microsoft, which allowed them to establish both companies as the essential players in the desktop computing world (the partnership often being recognized as the Wintel (Windows plus Intel) behemoth.

When the Internet started to rise, the network jeopardized that relationship as open standards offered the ability to move more of the software logic to servers and rely less on the client desktop, with HTML being pretty much the universal interface to those new systems. With the advent of Linux, a cheap alternative to Windows, Intel found itself remaining in a very strong position (as Linux can run on Intel boxes) and Microsoft sees the possibility of being increasingly marginalized. The problem comes from the fact that Microsoft, as holder of the software component is really only working as a middle tier in a relationship that involves processors, network bandwidth, software, and content. Let’s review why this development is significant in the new world.

Ten years ago, the big challenge in computing was processing power. Software was always coming out that needed to gobble up more processing power and more memory. In the last couple of years, though, the equation has shifted radically. Increasingly, users have more processing power on their desktop than they can use. Unless you are a hardcore gamer, the combination of Moore’s Law (which has pushed CPU speed to a point where any gain is of little relevance to most users) and the steady decline of prices for memory has meant that today’s user is finding himself/herself with a computer that is only gated by one factor: speed of access to the Internet. The challenge here is that, for most people, access to the Internet still happens over a regular modem, hence limiting what they can do online. While adoption of broadband access is growing, it still represents a gating factor in what most users can do. As a result, most people are now looking at how they can access the Internet faster, moving the discussion away from the desktop and onto that bit of the network that has traditionally been the realm of telephone companies.

With the rise of cable companies as access providers to the Internet, Microsoft now needs to find partners in two access camps: on the one hand, it needs to partner up with cable companies, and on the other, it needs to partner up with phone companies. For the first time in its life, Microsoft is actually forced to play in an arena where the monopoly players are somewhere else than in its own company.

With the AOL partnership, Microsoft is closing one part of the equation, by getting access to the pipes offered by Road Runner, the high speed access company offered by AOL/Time Warner. Coupled with relationship established with Verizon and Qwest, Microsoft has gained a foothold in the access space. However, this is potentially short-lived, as Microsoft could easily be replaced if any of those companies decided that they wanted to partner themselves with someone else.

So securing access to the pipe is one way to ensure continued relevance but it does not ensure the level of control that the desktop monopoly once allowed. In order to get that level of control, one must find a way to leverage the existing platform (windows) and create a lock-in with it. This is where partnerships on content can become useful.

In order to create a long term strategic control, Microsoft must ensure that it will be difficult to move away from its offering. This is where the Windows media strategy comes in. If Microsoft manages to get control of content created on the Internet, it will be much more difficult to unseat it in the future. With last week’s announcement that AOL would collaborate with Microsoft on digital media, the companies have started to establish something that may give Microsoft much more control in the future. Once content is encoded using the Microsoft Windows Media solution, it will be difficult to move away from it. A partnership on Digital Rights Management also ensures that Microsoft will hold the keys for content encoded using its solution, hence ensuring its tight control of a very lucrative market.

The ace card Microsoft holds in this is its installed base. By moving the dialogue from web servers (a battle it lost long ago) to video and audio servers (a battle that has yet to be fought), Microsoft is positioning itself for the future of the Internet. This early position will ensure that it will be able to offer Windows Servers that power the next generation of Internet content. The key in making its case is that, because it has control of the desktop, Microsoft can offer millions of users with a media player already running on their machines. This is an attractive public, and allows the company to make a strong case for an integrated suite of products and services (“here’s the player, here’s the server.. oh and while you’re using our streaming media server, how about using our rights management system… and you know all that stuff actually runs better on our windows platform…”)

So this is the worst case scenario. But, one can easily say, there are competitors and there’s no guarantee that this will work. Furthermore, the open standards are always creating a limit on the company’s power, right?

Well, that’s not even a guarantee. As we know, Microsoft came from behind in the browser wars. First, there was Netscape, and it was controlling 80% of the market. Then Microsoft launched IE but things didn’t really change much in the beginning. As Microsoft improved its browser (and Netscape, drunk on its own hype, believed it couldn’t be defeated), the percentage of control shifted.

AOL, with its established customer base of 30 million, and its ownership of the Netscape browser (bought as the company was already losing marketshares), was the only company that could have change the balance back. By bundling Mozilla first in Compuserve and then in the mac client for AOL, it indicated to Microsoft that this was something they might be willing to do, if Microsoft didn’t work with them. It quickly became obvious to Microsoft that they could be locked out of the browser market if they didn’t play nice with AOL. So they cut a deal and gave AOL a royalty free license to use the browser for the next seven years. That seemed to pretty much lock everything in place to keep tight control.

But the story doesn’t end here…

Apparently, Microsoft does not intend to build a standalone version of IE anymore. The relevant lines in that discussions are as follows:

Q: when / will there be the next version of IE?A: As part of the OS, IE will continue to evolve, but there will be no future standalone installations. IE6 SP1 is the final standalone installation.

Zeldman points out that IE will be built into future version of MSN for the mac but that otherwise, it will be part of the OS. This is an interesting development.

Let’s extend this concept out to beyond seven years: Microsoft and AOL are at the end of the current agreement. AOL did indeed use the Windows Media suite and is using the OS-embedded IE. Microsoft decides to renegotiate terms. AOL balks. Microsoft says that it will change its browser so that AOL doesn’t work on it. What happens then? What is AOL’s fallback position? On one hand, it’s got 7 years worth of media now encoded in Windows Media format (and would need to reformat all that in order to move off the Microsoft platform, a huge undertaking unto itself), and is locked into the Microsoft OS.

It seems that, unless AOL is keeping Mozilla alive, it is about to sign a deal that could eventually put it in a tough position on the browser end. It also seems that unless it hedges its best and encodes content in windows media and another format, it risks lock-in.

On the web development end, this also has huge repercussions. If we all develop solely to Microsoft, and agree to extensions they might make to HTML once its in the OS, we run the risk of all becoming windows developers, beholden to Microsoft.

This is a really all about a fight for the soul of the Internet. In the 90s, Microsoft announced a strategy of “embrace and extend”, which was often derided as “engulf and devour”. We’re now starting to see the extension happening, and it seems to point back to windows. Do we want to be locked in?

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Microsoft Lock-in?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the meantime — and possibly as a result– Microsoft and Verisign have just announced a new security procedure for person-to-person SOAP transactions, but a workable mechanism for securing Internet transactions between software and software may be years away.

Some of SOAP’s architects contend that building security into their protocol would only sacrifice its simplicity, and that the HTTP sessions that SOAP transactions rely on can already be secured at the session level, with protocols such as SSL. Moreover, securing sessions from outside interception, security experts believe, cannot protect transactions from two other perceived threats: interception from the inside and bad programming. With a protocol extension to SOAP for message attachments in the works, a third possible threat emerges — one that too many have become familiar with: malicious scripts.

Chris Dix, a SOAP programmer with FMStrategies, sides with the majority in believing that it may now be incumbent upon developers to endow applications with the specific security measures they need to communicate on open networks: If you opened up your [program’s communication] interface to be broad enough to accept things that might be dangerous, Dix says, then it would be your responsibility as a developer to make sure that the requests that might be dangerous came from people who knew what they were doing, and that you built in security.

Unlike the exchange of documents — spreadsheets and word processor files — between two people who can use public key infrastructure (PKI) or other measures to identify each other, distributed software components will communicate with one another without human intervention. In the new net services platforms such as Microsoft’s .Net, Novell’s One Net and Genuity’s Black Rocket, distributed software components will be everywhere, placing remote procedure calls (RPCs) to one another using the XML protocol. So why is the W3C close to deciding that security is not an issue, at least for them?

Should W3C Address Security Concerns?

The security debate began last May, when Ken MacLeod, an engineer of XML-RPC — a SOAP forerunner — published an article and posted a link to it in the W3C mailing list used by SOAP’s key engineers. While some rigorously developed applications may be thoroughly screened for security holes, MacLeod wrote, the vast majority of applications will never have security as a high priority. He went on to write that the syntax and content of RPCs are based on APIs, and that APIs are subject to frequent change. Every time an API is altered or amended, wrote MacLeod, security analysts would need to reassess the implications.

Many who began dismissing SOAP in the belief that it would be insecure, according to professional developer James Snell, author of a forthcoming SOAP book for O’Reilly, may have done so because it was originally marketed as a great way to do RPC.”

Speculation arose that SOAP could lead to a nightmare situation where one program could automatically hook deep into another program — and the owner would have no idea what had been done, and no way to prevent it.

There’s been a lot of concern it’s not a secure protocol because they didn’t define any security, says Snell. He explained, however, that security was never SOAP’s intention: It’s just an envelope for packaging data. In other words, you can’t blame an envelope for not being a safe. To reassure those who are still worried, Snell adds, Nothing in SOAP is automatic; just by using SOAP, your system doesn’t automatically open up.

Snell’s viewpoints are shared by many at W3C, including representatives of Xerox, who recently posted this:

Authentication, encryption and reliable delivery are already addressed at the level of protocols like HTTP and SMTP.

RPCs and the sessions that bind them are inherently complex, the Xerox engineers wrote, and any attempt by the XML Protocol to address these complexities would be redundant.

The XML Protocol is evolving into a way of using XML to encode data in a way that anybody can read it, no matter what operating system or language, according to Snell.

People should think more about the concept of interoperability than merely RPC. SOAP is more like a universal API. Without SOAP, you’re constantly writing specific APIs between applications — CORBA apps can speak only to CORBA apps, COM apps to COM apps. With SOAP, CORBA can natively interact with COM and vice versa. It’s an Internet standard way of communicating — no single company can get a lock in.

Securing Remote Procedure Calls

As the company best known worldwide for being able to get a lock in, Microsoft is recognized today as SOAP’s leading proponent. Microsoft promotes SOAP as a lightweight protocol for the exchange of both information and RPCs in a decentralized, distributed, networked environment.

The principle of RPCs dates back to Microsoft’s creation of the Component Object Model (COM), a way for small parts of programs (libraries) to be linked together as one program at the time the user runs the application, as opposed to the time the programmer compiles its source code. To move COM out of the confines of a single processor and over the Internet, Microsoft developed Distributed COM (DCOM), which let Windows applications make RPCs to other Windows apps.

Ironically, developers were originally attracted to SOAP, says Dix, because of some of the security nightmares they faced when trying to do DCOM over the Internet. It just was hard to get working, if you could do it at all. The security issues were just awful. CORBA had its own complexities as well. SOAP was written with the intent that people have to work inside of a corporate environment with a firewall, and need to be able to perform the sort of functionality. I know, the IT managers, as soon as you start talking about sending remote procedure calls as HTTP, get a chill down their spine.

SOAP’s dependence on HTTP and Internet port 80 as its primary transfer medium is a method that has been affectionately dubbed tunneling over firewalls. Although this sounds like a built-in measure for security breaches, Dix says, the technique actually relies upon most firewalls’ open acceptance of port 80 to get the message across. Because SOAP is XML and because it is transport independent, he says, it can be — and in the early examples, it has been — applied to sending messages as HTTP over port 80, and thereby circumventing some of the security issues with the firewall. Within the protocol and the specification, however, there are ways of identifying and using HTTP headers, and SOAP headers as well. As Dix explains, targeting SOAP messages for port 80 can enable content filters at the receiving end to scan for explicitly labeled SOAP messages — which could, if an administrator deemed it necessary, be blocked.

Here’s one example of a conceivably common SOAP session: A word processor could use HTTP to place a remote call in XML to a language translation application, requesting that its document be translated into a foreign language. The remote application would respond with an XML document containing the translation, in such a way that the end user would never be aware of the remote application.

How would these applications identify one another, and how is the exchanged data secured? Intentionally, SOAP by itself addresses neither question. Messages between applications are sent in the clear, meaning that anyone who intercepts the transmission and can read basic XML will have access to this information.

To protect yourself, advises Snell, you should at the very least encrypt the memo, as you would with confidential e-mail. In addition, you could send it over SSL, rather than insecure HTTP. Further, you could encrypt the SOAP envelope itself. A method for doing precisely that last item may have just arrived.

XKMS: A Solution In The Works?

The recent announcement by Microsoft, VeriSign and webMethods of a secure XML specification for digital signatures and encryption, called XML Key Management Specification (XKMS), promises to provide some security and peace of mind, at least for users. XKMS is a specification for managing public keys used to support digital signatures or encryption, or other applications of public keys, Verisign’s chief technology officer Warwick Ford tells us. So, it’s designed to work specifically alongside, and in conjunction with, the recent XML signature standard prepared jointly by IETF and W3C.

XKMS offers tools for digitally signing and encrypting documents shared between SOAP applications. So conceivably, a spreadsheet sent between computers could be both protected and authenticated, without engineers needing to amend SOAP itself. Although examples of XKMS for distributed object signing have yet to be investigated, Ford says, XKMS supports signing of XML objects…like business transactions. But it’s not limited to that. So, indeed, if you wanted to build some kind of software distribution system, which was itself an XML application, then you could use this mechanism for signing those objects.

Designing applications properly is the best way to minimize security holes, says Snell. It goes back to good application design — any application is insecure if you use it improperly. If a developer uses SOAP to write an application that accepts application code and then executes that code without first discovering where that code is coming from — ensuring that trusted relationship — that developer should be fired.

Dix suggests that security could be built into a SOAP enabled application by restricting the number of functions it exposes to the outside world — in developer parlance, by limiting its interface. Almost exclusively, says Dix, SOAP applications would not open up [broad] access to the components that exist on the server. Instead, he says, developers should adopt a very focused solution, one that was geared to exposing the functionality of one or a handful of components that you might have on the server.

As late as this week, proposals were being entertained by W3C to drop references to security measures in its upcoming XML Protocol draft, in favor of encouraging applications developers to build security into their own programs, and network administrators to monitor the communications channel. Whatever group provides the final answer to the XML Protocol security dilemma, it is now fair to assume that SOAP’s inner circle of engineers will not be part of it. Developers and security experts may have a rough job ahead.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Securing SOAP. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

AOL started moving further on its AOL Anywhere Strategy by announcing partnerships to deliver its Instant Messenger services on the Bell South and Sprint PCS network and to equip Neopoint, Nokia, and Motorola devices with the necessary software to do this too.

Microsoft announced partnerships with Nextel and Airtouch to deliver MSN to their networks. This follows recent announcements by Microsoft that its technology would be integrated in Sony and Quallcom wireless devices.

Meanwhile, Palm Computing announced a deal with Sun Microsystems to make Sun’s iPlanet service available to Palm VII users.

But with all the hype, one has to wonder whether wireless is truly here and what hurdles it has to overcome. From this issue on, I will take a quick look at some of the issues facing wireless web implementors these days, adding wireless as a new category of coverage. We will start with the formats.

WAP, WML, HDML, PQA???

It seems the wireless space in adept at developing a new set of standards. While this world is just burgeoning, a number of implementations have already surfaced.

WAP

: As defined by the WAP Forum, WAP is the Wireless Application Protocol. Think of it essentially as HTTP for the wireless crowd. Backed by the W3C, the IETF, and the ECMA, as well as most large wireless industry players, WAP has become the de facto standard for wireless delivery. However, some companies (NTT comes to mind) have tried presenting alternatives to WAP and have so far been relatively unsuccessful. However, I doubt that WAP will go very far as it limits the number of characters that can be sent to about 1600. For stock quotes or weather reports, it’s a great think but beyond that, I doubt that anyone will use it for Ecommerce or content.

WML

: WML stands for Wireless Markup Language and is an XML based subset of HTML. However, a war as broken out in that space, with phone.com (one of the early pioneers in the wireless space) striking out on its own and developing a competing standard called HDML.

HDML

: HDML, or Handheld Markup Device Language, phone.com proposal for a new markup language. At the current time, the W3C has worked with phone.com and other markup language partners in an attempt to resolve the incompatibilities between the two offerings. With the cachet of WML increasing over the past year, phone.com has started supporting both format but offers HDML has a language with new tags that allow it to extend WML applications. Because it was an early player in the field, phone.com has taken a lead and could be the Microsoft or Netscape of that space. As a result, the extensions they are providing can’t be ignored.

HTML 4.0 mobile

: Last year, with the introduction of HTML 4.0, the W3C made some recommendations in terms of supporting HTML for wireless devices. Throwing further confusing in the wireless space, the W3C decided that HTML 4.0 and its successors might be the way to go, throwing more oil on the wireless fire. While no recommendation has been made yet on an actual standard and in spite of the W3C’s claim that it is working to resolve disputes with the W3C, expect some serious in-fighting between the different groups as they try to position themselves in the next hot web application space.

PQA (Palm Web Clippings)

: A couple of years ago, I pointed out that the Palm OS could be a potential Java competitor in the non-PC devices space. As could be expected, Palm went out and introduced the Palm VII, a wireless device with connections to the web. What was surprising, however, is that instead of going out and supporting either WML, HDML, or even HTML, they decided to introduce their own format to distribute web content: PQA or the Palm Query Application language. PQA is a paired-down HTML version that allows you to distribute content on the wireless Palm platform. Since services like OmniSky plan to offer wireless access to Palm devices other than the Palm VII, and since Palm already has an established footprint in the PDA space, expect PQA applications to pop up left and right.

A lot of format but what do I implement for?

At the current time, it seems there are no clear winners in the space however it seems clear that WAP has a strong lead in the delivery space for small bits of data. But WAP will not be the way to do Ecommerce or content as a clear character limitation makes it fairly useless for this. On the markup front, I’d strongly recommend looking at WML as it has received support from some of the larger players (Microsoft and Sun, among others) and seems to be the basic level of functionality. However, you should also look very seriously at the PQA format because of Palm’s extremely large footprint in the PDA space.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Wireless: A confusing Landscape. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

RSSmanip is a set of scripts that allows webmaster who already generate an RSS feed for their site to dynamically create HTML, HDML, and WML documents based on their RSS feed. Using the Microsoft XML parser (supplied with IE 5) on the server side, you can now enable your site for wireless usage by using those scripts.

Minimum Requirements

I haven’t tested these scripts on multiple platforms but here’s the environment I used to create them:

• - NT 4.0 with service pack 6
• - IIS 4.0
• - Internet Explorer 5.5 (this is important because Microsoft replaced the XML engine on which this script relies)
• - A well formatted RSS file

What files are in RSSmanip.zip?

Recently, I’ve started doing some manipulations or RSS file for different presentations. Included in this zip file are three files to do so:

• html.asp Creates an HTML output of your RSS file
• hdml.asp Creates an HDML output of your RSS file (for more on HDML, check out http://www.phone.com)
• wml.asp Creates a WML output of your RSS file (for more on WML, check out http://www.wapforum.org)

Installation

Installing those files is relatively easy.

• Download the source code here.
• Open each of them in your favorite text editor and edit the following lines: mylogo = "./presentation/images/TNLwlogo.bmp" myRSSfile = "/newsletter/channel.xml" mylogo is the location of your logo file. For HDML, that file has to be a .bmp file and for WML, it has to be a .wbmp file. Editors for each are available on the web.myRSSfile is the location of your RSS file beyond the root.
• Open up the IIS console, go to your server properties, click on HTTP headers, click on File Types and add the following: .bmp image/bmp .hdml text/x-hdml .wbmp image/vnd.wap.wbmp .wml text/vnd.wap.wml
• Save the settings and reboot your server.
• Put the files on your site and that’s it. You’re up and running.

• Optional:

  If you want to redirect WML and HDML browser automatically to the appropriate files, use the following script: acceptHeader = Request.ServerVariables("HTTP_ACCEPT") If Instr(acceptHeader, "hdml") 0 Then Response.Redirect "/hdml.asp" Elseif Instr(acceptHeader, "wml") 0 Then Response.Redirect "/wml.asp" End If where /hdml.asp and /wml.asp are the locations or those files (in my example, they’re in the root directory of the server.

Other Version

An earlier version of these scripts allows you to generate code from a RSS 0.91 feed. It is also available for download.

How much does it cost?

RSSmanip is free! If you want to use it, I’d like to receive a link back to TNL.net from you or receive a donation from you.

Final comments

I’ve created those files for my personal use and am happy to share them with you. However, I do not make any guarantee as to their impact on your server. They seem to work fine on mine when I used them. If you use them, I’d appreciate a link back to TNL.net. Since I moved TNL.net to a Linux platform in late 2002, I am no longer supporting these scripts.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title RSS Manipulation in ASP. You can follow him on twitter here or receive his weekly newsletter by subscribing here.