The Open Graph

Before we look at the winners and losers, let’s recap where Facebook is going. A couple of years ago, the company unveiled the “Open Graph API”, which gave external sites the ability to add “like” buttons to their sites, and in the process allow end-users to things they like into their facebook stream, where they would be shared with their friends. This mean that a user could implicitly publish information to Facebook by just clicking a button. At that point, two things really happened:

• First, Facebook created a system that allowed it to better understand what people were reading because every time a like button is pulled down, Facebook can keep track of it and associate the information with the account of a Facebook user.
• Secondly, Facebook offered publishers a way to generate more traffic to their site by publishing content on Facebook that would return users back to their site.

This was a little creepy because it gave Facebook a tremendous amount of power but, in exchange, the company gave publishers access to more traffic so the agreement seemed balanced to everyone but the end users (which doesn’t matter, as we all know that “if you’re not paying for it, you’re not the customer; you’re the product being sold.”)

But it seems having access to that data is not enough for Facebook; now it wants more data so it can build better electronic profiles of its users. So this year, the company has decided that “Like” was not enough and they wanted to get users to give them access to anything they read on a particular site. So the company unveiled actions like “read, watch, listen” which allow developers to share all action data from a user after a one-time sign-up to the Open Graph API.

Mining the web… but not giving back

This time, the company is mining the web but the main beneficiary of those actions seems to be Facebook, which gets a better understanding of where users are when they are not on Facebook and what they are looking at. If we assume (and I am willing to take the bet that this will be the case) that most mainstream sites will start offering the new verbs, then Facebook will have one of the most complete understanding of a user’s demographic and psychographic profile. In other words, the data it will get access to is the holy grail of online marketing: users that can be tailored to based on extremely granular preferences.

That data can then be used to send ads that resonate with the user in the channels the user accesses most. Think this is crazy? Well, wait ’til next year’s F8, when Facebook unveils a tool to help external sites monetize their traffic better by targeting advertising based on users’ preferences. If this sounds suspiciously like AdSense from Google, it’s because it is part of the end-game. Facebook is no longer happy to have the largest site in the world, now it wants to have access to people when they are not on Facebook.

What’s fascinating here is that the data goes into Facebook but there is precious little information as to how to get it back out, making the word “open” a headscratcher as it is unclear how Facebook defines openness. To the Palo Alto company, it appears that openness is a one-way street: you open up your data to Facebook and in return Facebook “simplifies” the online experience by keeping your app on its platform. This is somewhat similar to the app store model offered by Apple and Google, where it’s OK to play as long as it is within their rules. Facebook is doing to the open web what Apple and Google have often been accused of doing, sticking another knife into its imminent demise.

Also of note is the fact that Facebook’s approach to getting all this data makes it impossible for anyone to create valid HTML5 pages as the Facebook code does not validate under this framework. So Facebook is also hampering the future of the web by making it nearly impossible to live by the ideal of the new web standards if you want to play in the facebook arena. This seems to shape up another fight between Facebook and the open web.

Another company has had similar ambition and it staked its approach first on offering superior search products and then on using the search data to target advertising on partner sites and eventually offer such capabilities to anyone who was willing to give them a percentage cut of ad revenue. That company, Google, has realized the limitations of its model and is busy trying to ensure it can get more data by building up offerings for ways in which people access the internet: so they’re pushing Android for mobile phones and Google Chrome as a better web browser because they want to be able to access data relating to where people are on the web, data that can then be used to create more customized ads.

Facebook at $150 billion?

This week, Google’s valuation sits around $150 and the highest Facebook has ever been rumored to be worth is $100 billion. I’d venture that people are selling the company short and that it is worth something on par with Google. It has masterfully played fears from online publishers and other sites and parlayed that in a potential position of power in the online marketing world.

The only thing that could make it more powerful than it is about to be is if it were to pair up its data with Google’s. A merger of the two of them would create an unparalleled database of internet users, containing just about anything about people’s intents (from Google’s search), their interests (from Facebook’s data), the amount of time they spend on certain properties over others (from either Facebook or Google’s data), and what they liked enough that they would share it with people their know (from Facebook’s data).

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title The “Open” Graph. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

It has a relatively low but highly dedicated audience and is garnering good press both in the blogging community and the mainstream media.

The service is suffering from growth related issues which force it to be down at unexpected times but users put up with it because of its supposed transformational nature.

The service allows people to build things on top of it, offering external parties a greater chance to generate revenue than the company providing the service.

And, establishing further proof that service is going to be important in the future, a lot of mainstream stars are establishing presence quickly, only to slowly abandon those points of presence after a while.

But those stars are no different from most of the service’s users, which tend to abandon it only a month of two after trying it out.

What is that service called?

If you said Twitter, you are clearly reading this in 2009. But, only two years ago, the answer would have been Second Life (something I learned first hand, having been part of the hype around it back then).

of course, I have no doubt that this post will probably receive a high amount of flames because supporters will tell me how Twitter is different. But is it?

The Coral Reef

I’ve always had an affinity for Dave Winer’s Coral Reef analogy. However, even the coral reef analogy seems to eventually break down, leaving people like Winer to think of ways to move out (in a way, Winer fell into the same trap with Twitter as Scoble did with Friendfeed).

The issue here is that a lot of energy gets poured by developers into supporting an ultimately closed system. While artificial coral reefs exists, they are generally part of the larger ocean and tend to be pushed into creation by sinking boats or subway trains. But an important distinction is that the creator of an artificial reef is generally present at the creation but then lets the ecosystem take over and doesn’t try to control anything.

In the tech field, the best analogy for an artificial coral reef would be opening sourcing an important source of code (for example, the apache web server) or making a set of protocols or ideas open to all (eg. HTML or RSS) without requiring that the implementor cede any control to the party which made the code or idea available. Today, you can fok the httpd server if you feel like it or you can adapt parts of HTML or RSS to your heart’s content.

Fauxpenness

But there’s a different set of ecosystems out there that becomes more of a venus flytrap of technology. I would describe this as fauxpenness:

Fauxpenness: Calling a system or platform open while it is, when more closely scrutinized, under the tight control of its provider.

Fauxpen system (or fauxpen platform): a system or platform that claims to be open but, upon closer examination, isn’t.

It’s the kind of approach that pretends to be open but provides some level of lock-in.

In 2006–2007, we saw that happen with SecondLife, as many developers (myself included) built software code that could run within the SecondLife world but was ultimately stuck there because you could not run it outside that world and/or run SecondLife servers on your own machines.

in 2007–2008, we saw that happen with the F8 Facebook platform, which locks your applications inside of Facebook and, while many developers have pushed to force the company to open up, tends to stay there. In 2007-today, we’re seeing the same thing with Twitter, which allows you to build whatever you want on top of it but doesn’t decentralize their approach, leaving developers potential slaves to the whims of the company. The same is true of the iPhone, which provides unusual access to the phone operating system and allows to develop interesting software on top of it but still keep developers away from being able to access basic things like calendar information via an SDK.

The endless cycle

Interestingly enough, it’s not an unusual phenomenon in the technology world. It works like this:

It happened with SecondLife; it happened with F8; it will happen with Twitter and it will happen with the iPhone at some point. It appears that the natural course of locked API is to get to a point where the developers get so annoyed that they decide to go look somewhere else.

But there’s hope.

Breaking Free of Fauxpenness

Because of the lock-in, it is possible for companies to break free of the cycle. In order to do so, two things need to happen:

• First, the company needs to find a way to establish a business model that does not require lock-in
• Then, the company needs to start removing the lock-in components it offers.

I’m not saying that either of those step is an easy one. In fact, few companies have successfully managed them and, even when they do, the developer community will keep asking for more.

For example, Microsoft’s history is one of establishing initial lock-ins, weeding out the competition and, when its lead is established enough, relaxing the choke-hold it has on the developer community and playing a little nicer until it tries to enter another market. That was the case with Windows; it was the case with Office; and it is the case with IE today.

IBM also took the same approach, initially being a provider of proprietary systems and slowly, over the last 15–20 years, moving to become one of the largest supporters of the open source movement.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Fauxpenness. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Before I go any further in this, I want to have a huge disclaimer: I’m a pretty big fan of some Google products. One can see Google ads running on this site (I’m an AdSense user) and a portion of my traffic gets here thanks to Google’s search engine. i’m also a big user of the search engine, I have a Gmail account (although it is not my primary email system) and I use Google Maps and Google News often. I’ve played with the search API in the past and, for the most part, I’ve been happy with my overall Google experience. However, I worry that the company is heading in the wrong direction and I want to ensure they remain a viable player as they have re-ignited investments in the search space, which has benefited all users on the Internet. However, I fear that, if they are not careful, they could suffer a fate similar to that of Netscape, which popularized web browsing and ended up being gobbled up by AOL, where it is now a shadow of its former self.

That said, let’s look at some of the disturbing similarities.

Market Shares are no guarantee

In the early days of the commercial Internet (let’s say 1996), Netscape was a very successful company. It had beaten every Wall Street expectation and completed a stock offering that had captured the imagination of the general public. The Netscape management graced the covers of most magazines in America and the little browser that could (then in version 2.0) had captured an impressive 75+ percent of the market. Netscape had also introduced its own line of web servers, with a proprietary language called LiveWire, which allowed to create more dynamic applications. The company was also offering a web page development tool, and struck partnerships with many companies to integrate their audio and video components with the browser.

Microsoft had come out with Windows 95, which included a browser (Internet Explorer) which they had licensed from an outside source (NCSA, the place where Marc Andreesen had worked prior to Netscape and the browser was Mosaic, an early web browser Marc had been involved with). The world had mostly laughed at the pitiful version 1.0 offering from Redmond. It was simply a bad product, which did not get much redemption with version 2.0.

Microsoft on the Offensive

The folks at Netscape were feeling pretty smug. After all, they dominated the browser market, had managed to get a way to sell server products and comments about the upcoming irrelevance of Microsoft started making the rounds. But the giant was awake and the clouds over Redmond only covered a flurry of activity. By the time IE 3.0 was released, most people had written Microsoft off. If they couldn’t get as simple a piece of code as a browser to catch up, how could they have a chance to survive.

Netscape had come out with version 3.0 and it was good, if a little bloated from the everything but the kitchen sink approach they were taking. Netscape was now offering an Internet suite that included a browser, a mail client, a newsreader client, an IRC client, some groupware capabilities, etc, etc… There was no way Microsoft could catch up.

Netscape Navigator 4 came out and it was good. It was running Java applets, it could do DHTML, etc.. Basically people liked it and didn’t see a reason to switch…

But Microsoft released IE 4.0 and it was better than people expected. it matched the Netscape browser feature for feature and threw in a few things. One of the people in charge of that development was a guy by the name of Yusuf Medhi, who now happens to be the head of MSN.

While Microsoft had fired a major shot with that new browser, everyone expected that all that would change again when Netscape 5 would come out.

Netscape 5 never came out. In fact, Microsoft release IE 5.0 and started gaining market shares (stealing them from Netscape). Netscape seemed to be trapped in its own legacy and had problem getting a new product out. Microsoft release IE 5.5 and Netscape was working on a new rewrite of their product.

Finally, Netscape 6 came out, conveniently skipping a version. Was it the answer to Microsoft that all had hoped? Not quite and by that point it was too late.

Netscape never recovered and now lives as a shadow of its former self. Microsoft put out a 6.0 version of their browser, cleaning up some of the last parts of the markets they wanted and then went to sleep, in terms of browser, until the recent competitive threat of Firefox reared its head, eating up some of their hard earned market shares.

So what went wrong? The answer is complex but I believe that a mix of Hubris (we can beat Microsoft, we have a huge market share) combined with some sloppy releases, the development of a bit of a monoculture (we set the agenda, the industry will follow), an unwillingness to deal with massive competitive threats, a loss of focus on core assets, and a media world that loves to take down the companies they’ve built up all added up.

How does this apply to Google?

For starters, it is clear that massive market shares are no guarantee of success. Google currently holds around 60 percent of the search market, which is good but is also a reason for concern as it is more likely that this share will go down than it is that it will go up.

More worrisome, however, is the development of the Google monoculture. Much of what is going on at Google is happening with little involvement and input from the community. This is where Microsoft generally starts striking. Say what you want about the Redmond giant, they know how to listen and how to take brutal feedback and turn it into decent product. Microsoft is not known for great products but it is known for decent ones. Last week, Microsoft organized Search Champs, gathering a bunch of smart people from the industry in a room and having them talk to them. I was there and was surprised by how focused they are on winning this one. It is the kind of focus I have not seen come from them since the browser wars.

If it wants to survive, Google needs to do something similar. Throwing a product out to the world with the world beta on it is not a feedback loop. Sitting down with users, developers, thought leaders is. The feedback is not always good but it helps improve the product, which is how one wins this war. Furthermore, the goodwill generated by getting people invested in its products and their success allows a company to develop a strong following from a small group of dedicated users, who then serves as advocates in the marketplace. They can have an impact in changing opinion and not involving them can be dangerous.

Of those people, developers tend to be the more finicky. Alas, the success of many platforms on the Internet depends on developers. As developers go, so tend the marketplace because developers tend to be early adopters. Developers were the first people to switch from Yahoo to Altavista. They were the first group to switch from Altavista to Google. Where will they go next? Is it guaranteed that they will stay with Google (however, here is an interesting case, as developers tend to have a bias against Microsoft. The corollary of this is that Microsoft has to offer something that is radically better in order to make gains in the developer world). A good way for Google to mend some of the rift with the development community would be to support RSS along with ATOM as a syndication format. At the current time, Google is the only major search engine without native RSS support.

Another area to watch out for is the loss of focus. Could someone at Google please explain to me how the Google pack, Google WiFi, Google IM or the Google web accelerator fit Google’s mission (to organize the world’s information). How does owning a radio advertising business (something they acquired recently) fit in that model? It seems that Google is trying to do a lot of things in a lot of areas. I’m sure they’re all interesting things but what does that do to the core search assets on which the business was build (or is it that search is just a side business and Google’s mission is really about advertising?) There has been much discussion in the search world about the relevancy of results in the Google search engine suffering from some level of degradation. As always, expectations are high and any decrease (or lack of improvement) in the quality of the search index will be seen as a loss of focus.

Following the Netscape sloppy release, Google also has to worry about better testing before putting products out. Its recent stumbles with the release of Google NewsReader and Google Analytics showed the world products that were not fully ready for market release. The market acceptance for the word beta goes only so far and Google may suffer some reputational damage if it continues along a curve or release first and fix it later (this, however, is not necessarily a standalone cause for failure, as we’ve learned from the release of many Microsoft products that needed their own round of stabilization)

Last but not least is the burning glare of the media world and of Wall Street. As can be seen now that lofty (and, one could add, unrealistic) expectations could not be met, punishment (in the form of a declining stock price) is coming. Similarly, the press is getting more critical. This is part of a normal cycle: a company is hyped up and then taken down. These are just fads (ask your friends at Yahoo!, who have managed to go through the whole cycle and are starting to go back through a build-up phase now).

And, as a postcript, take the advice of pundits like myself with a grain of salt. There are lessons to be learned but I can’t guarantee that these are the right ones to learn. However, what is certain is that Google needs to remain a viable player in search if for no other reason than to keep companies like Microsoft honest. As we’ve seen in the browser wars, once a company wins, it tends to slow down on the innovation front and search is still so young a field that it needs major progress on the innovation front.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Googling Netscape. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Grouping the metrics

In order to figure out weighting, I first started to think about how to group different metrics. For this purpose, I looked at things like the base value (which would give us a baseline as to how much a business is worth based solely on revenue and revenue growth), inventory (looking at things like traffic, reach, and output, because they all give us some data points as to the growth of monetizable assets in the future), consumer involvement (looking at info like links, subscribtions, and comments to define the value of customers), and growth potential (including some more fuzzy measure of potential growth and the advantages of the integration value).

My reasoning for grouping things in this way was that it might make it easier to figure out weighting across those large catch-all categories (and, if there is any discussion at all, I am sure that people will debate the percentage assumption against those categories). I, in no mean, try to represent those as the be-all-end-all approach to valuating a business. They are, at this time, the metrics that give me the best comparative view of a business, when I try to assess its value. However, not being much of a metrics guy to start with (my main reason for doing this series is to provoke debate among people smarter than me so there can be some consensus on metrics in this new web 1+n.x world), I hope that others will step in and show me the error of my ways along with providing some interesting information that will get all of us closer to something useful.

Base Value

The base value, as I see it, is defined by revenue and revenue growth based on historical data. The reason I would consider this to be the base value is that it is a reflection of the business as it exists today and can provide a baseline as to where the business would be headed if growth suddenly slowed or investment in the business stopped. It does not provide any information as to how to accelerate the growth of the business and does not provide more than a view into the present cash value of a business.

However, for young companies, such value does not provide much information. Start-ups, by nature, have a lower revenue and profit line than established companies because they need to recover some of their initial cost and may still be in high growth and research and development phases. As a result, to solely base one’s view of a business on its current ability to generate cash is short-sighted when it comes to start-ups.

Another question when developing the base value is how to factor in risks to the revenue base. For example, if the business relies primarily on advertising from an external network as its basis for revenue (many people have talked about businesses looking to AdSense as the primary source of revenue), one has to wonder what would happen if the dynamics of that relationship were to change.

As a whole, however, because of its overall importance in assessing the present financial value of a business, I would assume that the base value should represent about 20 to 30 percent of the overall value of a business. Initially, the value would be in the 20 percent range because potentials are higher than the current revenue line but, as the business matures, and potentials decrease, it would edge up towards 30 percent.

Inventory

Inventory would be the next potential grouping of different metrics. In it, I would include traffic (and traffic growth), as well as visitors, site counts, reach, and output. Let’s go into more details on each of those.

Traffic is important because the number of page views is something that is monetizable. However, in a web 2.0 space, pageviews are not the only traffic metric one should track. For example, RSS subscriber counts is another useful value (and controversy has already swirled around ads in RSS feeds). However, I would argue that there is one value in the inventory count that is of utmost importance: access to an API. The reason I would venture this is the most important inventory metric is that APIs, once implemented, are harder to unhook from. As such, they represent a harder type of value since they solidy a site’s reach within a particular market segment.

I believe that reach is actually going to be seen as one of the more important values in terms of inventory. The reason is that reach gives us an idea of the potential growth opportunity in a market. If a company has a high reach in an individual market, its potentials are more limited. Witness, for example, a company like Netscape, which once had a reach of 80% (ie. 80% of all internet users were using it. ) Tactically, this kind of position is one where they should have been on the defensive, the reason being that there was more potential of a drop in their reach than an expansion of it. Microsoft is now finding itself in the same position on a number of fronts: Windows, Office, Internet Explorer are all playing in a world where they will not reach a higher percentage of the market. As a result, they are forced to play defensively. One could argue that web 2.0 companies, with their reach APIs and more powerful front ends (thanks to technologies like AJAX) are representing the threat Microsoft saw coming from the Internet in the mid-90s. And one could argue that, this time, the position they’re in (ie. largest player) is endangering their future if they don’t make a radical change (because they can’t grow from the position they’re in).

Going beyond the reach, which provides some information on past growth and potentials moving forward, one has to look at output from the company. For example, in the case of a company like Ebay (arguably a web 2.x company already), the inventory is number of auctions submitted. Similarly, in the case of Craig’s List, it would be number of new ads posted, or in the case of a blog, the number of posts created. One has to be careful about ouput, however, and should measure the cost of output in order to figure out whether the output is good or not. In the case of web 1+n.x companies, output is a very good thing as it is generally created by outside parties for free. That free product is one that those companies then monetize. However, one has to be careful and evaluate if output is outpacing the company’s ability to monetize it because, if an imbalance were to start existing, the value of the output could potentially decrease.

All and all, because inventory has a measurable value and, in general, is the very thing that a company will monetize, I would guess its weight, when figuring out the value of a company would probably sit in the 10–15 percent range.

Consumer involvement

Consumer involvement, which was known in the past as stickyness, is another major group of metrics. This section would include links, subscriptions (both to RSS or API feeds and, if offered to any paid type of service), and any type of interaction a user may have with a system. For example, if you trying to get some interaction information on consumers of a blog, one could look at numbers of comments posted. Alternately, if you’re looking at a search engine, one could look at number of searches performed. Or, if you’re looking at a company which offers an API, you could look at the number of times that API has been integrated in other products and the number of times it is accessed.

I would venture to say that this metric is one of the most important ones when assessing the value of a business. The reason I would value it higher than the ones I mentioned earlier is that this is where one can see whether a business has a potential or not. The interaction with customers (either directly or via APIs) provides so much useful information that a company not looking at this metric is probably off track in terms of evaluating itself (and, generally considering the hype around new businesses, such company could fool itself into extinction as it fails to see major issues arising out of the increase or decrease in consumer involvement.)

Because it represent the value of the existing cutomer base and provides some input as to the trends surrounding that customer base, I would throw a weight of 30–35% of an overall valuation going to factors relating to consumer involvement.

Growth potentials

However, metrics in and off themselves, are pretty useless as a point in time number. As a result, one has to assume the growth potential when evaluating a business. The growth potential can be associated in a number of ways but, when it comes to web 1+n.x properties, it comes down more on the side of potential based on a number of subcomponents. In the interest of provoking more controversy, I would venture that there is a formula to calculate potential and that it is as follows:

Potential = traffic growth rate * reputation vector * brand equity vector * (integration vector (squared)) — ( Risks vector / percentage of risk that can be mitigated)

In the growth rate area, I would put an aggregate growth rate that averages out growth rates over a period of time (6 months to a year if you are computing a monthly growth rate.) The reputation vector and brand equity vector would be values based on reputational and brand equity trends, which I talked about in a previous entry. You will notice that I consider the integration vector to be of such high importance, when defining potential that I’ve decided to square its value. I will talk about integration vectors in a future entry but, put short, the integration vector is the magic glue that makes acquiring or merging a company very valuable because integrating it with another company will derive greater value for the combined entity. It is that issue generally known as synergy but trying to put a value on it would have the potential of making for better, more successful acquisitions and mergers. Last, but not least, is the rist side of the equation. Because risks have a huge impact on potentials, it is important to measure them in order to get an idea as to their potential impact. However, because some of the risks can be mitigated, it is important to capture this figure in order to assess the importance of different risks.

Ultimately, growth potentials represent the largest part of any equation when trying to value a company. Few companies are bought without an expectation of potential and this is why, in my weighting, I would assume potential to represent a substantial (30–40%) part of the equation when trying to measure a company’s value.

Conclusion: Wait, that’s more than 100 percent

If you do the math, it appears the different weight are ending up representing more than 100%. The reason is that those are ranges. However, the truth is that, in any business dealing, there is also an amount of faith and luck that comes in. For example, I sat in a meeting once where an individual was given an option to buy in whole a company which is now very successful on the Internet for around a million dollars. Looking back, it might have been worth that much at the time but I doubt that it would be worth what it is worth now (several billion dollars) had that deal being consumated. Over time, the management of that company was smart enough to mine opportunities and put people in place that helped them realize huge growth. Had that company been in the hand of more conservative (and by conservative, I mean adverse to risk) investors, it would probably not have flourished in the same way.

Having gone through a few days thinking about metrics, it is clear that there are a number of opportunities for people smarter than me to figure out some solid metrics in assessing the value of new companies. Metrics, however, should not be the sole guide when assessing a company.

Many people have asked me why I bothered to look at such boring subject (and why I’ve been blogging so incessantly about numbers lately). My main reasoning is that one of the failures in Web 1.0 (the bubble we lived through in the 90s), lack of accountability and/or expectation management lead to very inflated numbers that eventually left a lot of investors with very poor investment. Having lived (and survived) that bubble, I want people to start thinking more critically about Web 2.0 companies and, hopefully, we can all learn about the mistakes of the past and avoid over-hyping new companies into extinction… because for every bubble, there is eventually a big pop, and no one really enjoys that part.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Metrics — Weighting the Metrics. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

First of all, let me say that while it is an interesting package, it is not without flaws. For starters, the lack of indicator when the Spam folder and/or trash have stuff in them is a bit disturbing. Sure, it’s not that much of an issue when you have a gigabyte of space but, for those of us who are particularly clean when it comes to online operations (I generally try to keep my virtual trash relatively empty), it’s a bother.

On the spam blocking end, Gmail does an OK job but is nowhere near as good as simply using Squirrelmail with SpamAssassin, or using the spamcop black-list. On the good side of this is the fact that, in the course of a day of use (about 600 messages, most of them spam), Gmail had no false positives in terms of tagging things as spam. On the bad side, it failed to tag hundreds of messages as spam. I’m assuming that Gmail has a learning curve, like other email packages, so I wouldn’t fault it yet on this but, looking at other packages, it does not pass the test in terms of being an efficient spam-blocker.

The concept of conversations is an interesting one and I can easily see how this kind of interface paradigm represents the next step in the evolution of email interfaces. It is clean, simple, and adds tremendous value, especially when tracking discussions across mailing lists. Combined with labeling, it could become a good way to manage mail.

Speaking of labeling, I was disappointed to see that every non-spam email lands in the inbox. If I set a label and/or rule against a piece of email, I would like to see that view move to a different view/folder than my inbox. Once again, under the edict of keeping things simple and clean, I generally prefer to have a relatively uncluttered inbox, with emails being filtered left and right into different folders/views.

Another thing that annoyed me with the interface was its heavy reliance on JavaScript. There should be a Gmail Light version which does not require those tricks. This was most painful when I tried to access Gmail via my Treo 600. Since I don’t always sit in front of a full fledged PC, I can’t always use JavaScript and this means that, until Gmail offers either a way to access mail via pop3/imap or a non-JavaScript browser, my Gmail account will not be the primary one. The other problem with this is that certain browsers, like the popular Safari browser, are not supported. That’s a problem for those of us that do not believe in staying with one platform and/or browser. I tend to use Mozilla when on a PC (kudos to the Gmail team for their support of that browser), Safari on the mac, and Blazer on the Treo and in this day and age, I expect web sites to work on all three.

All and all, Gmail is an interesting email package but I expected more. Maybe it was the hype; maybe it was because it’s a Google product. Either way, I’ll be following its progress over time.

moving forward, I’d like to see the Gmail team develop an API (using SOAP or XML-RPC) so a developer community could start adding features or building on top of Gmail. It seems that this is another area that Google should investigate.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title The day I broke Gmail. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Considering the fact that Google already has both an XML-RPC and a SOAP bridge, it seems that this kind of service would be a no-brainer. Here’s how it would work: A message would be sent out in XML to either of the service and would include the title of the page, its URL, a status code (either new, updated or deleted), and specify the level of crawling you want (page or site). The Google service would then take this info and crawl the specific page or site as required.

The advantage for Google is clear: sites that use this service would no longer be part of the regular crawl for Google spiders, which could help Google focus on spidering other (new) sites that are not on this list. For sites that are running Google ads, this would also allow Google to produce better targeted results.

The advantage for web site developers is also clear: this system would allow them to more quickly get into the Google index, a pretty good source for traffic.

In the end, it’s a win-win scenario for all involved. So when will we see this new feature from Google?

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Google Ping. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Discoverability and Security

The first assumption is that software should be able to connect automatically. While this is generally a good idea, there is a need to set up different levels of accessibility. Businesses generally will want some level of controls over how accessible a machine is. From here, one must establish a level of trust to handle relationships between machines. This should include some basic categorization, user being the lowest level of categorization, then raising up to different groupings. For example, a user could be a member of a team, that team could be a subset of a larger division, which itself would be a subset of a company, which would be a subset of an industry. The idea here is to automate the process of identification and still ensure a degree of trust in order to maintain security. While a piece of software can be network aware, the network may not necessarily want it to be aware of all resources. For example, if I am a visitor at BigCo, BigCo may not want me to have full control over all their resources and full access to all their services. My machine should broadcast my credentials and, based on that, have access to certain resources.

Why Buddy Lists are NOT the way to go

Tim advocates the use of buddy lists to set up those relationships. I would venture to say that buddy list do not provide the level of granularity required. From there, there are two potential ways to go: enhance buddy list to allow greater levels of categorizations or come up with a completely new format. I would be tempted to go for the former as it is building on top of an existing standard.

Two way data and XML formatting

Tim makes a point that every piece of software should expose some version of its data as XML feeds. While I generally agree that the data should be represented in a common format, XML being the ideal choice, I object to it being a feed. What applications should provide is an API that gives access to that data instead of a feed. The reason for the semantic disagreement I have here is that a feed is generally pushed or pulled on a regular schedule, no matter whether it is needed or not. Providing an API would ensure that the data is only obtained upon request, therefore conserving precious network resources. A good example of feed misuse was Pointcast, a software client that would poll the network every few hours for feeds. The problem was that it would do so at the same time for every client on the network, thus generating network traffic spikes on a regular basis, and generating much hatred from network administrators.

A proper API could be designed using either XML-RPC or SOAP as a way to carry its messages.

Where does the data go?

The other issue is where the data should reside. As a general rule, computers are no longer well suited as the only repository of data. There is a need to represent data in a fashion that makes it largely independent of the platform it’s running on. A large part of the problem here is who you trust (there’s that trust issue again) with your data. For example, buddy lists in AIM are stored on the  AOL servers. Do you trust them with that data? Would you trust them with more personal data (written documents, etc..) ? Would you trust Microsoft with it? Would you trust anyone?

This brings up interesting possibilities in terms of either keeping the data on a single computing device, from which it might be shared, or moving it in a lot of different places (making it more difficult to ensure change control and general data management). This is an issue that still needs to be resolved.

Online/Offline

The one point that Tim does not cover is the online/offline challenge. One cannot assume that a computer is always connected to the network. As much as we would like it to be that way, computers are often disconnected from a network, whether it is on a plane ride, or when in a place where network resources are limited or inexistent. Programs should be aware of that state and still be able to work properly when offline. As a result, software should have a mode that allows it to check whether network resources are available or not. If they are, it should check the sharing arrangements. If there are none, it should still provide basic functionality.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Tipping the Edge. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

In the meantime — and possibly as a result– Microsoft and Verisign have just announced a new security procedure for person-to-person SOAP transactions, but a workable mechanism for securing Internet transactions between software and software may be years away.

Some of SOAP’s architects contend that building security into their protocol would only sacrifice its simplicity, and that the HTTP sessions that SOAP transactions rely on can already be secured at the session level, with protocols such as SSL. Moreover, securing sessions from outside interception, security experts believe, cannot protect transactions from two other perceived threats: interception from the inside and bad programming. With a protocol extension to SOAP for message attachments in the works, a third possible threat emerges — one that too many have become familiar with: malicious scripts.

Chris Dix, a SOAP programmer with FMStrategies, sides with the majority in believing that it may now be incumbent upon developers to endow applications with the specific security measures they need to communicate on open networks: If you opened up your [program’s communication] interface to be broad enough to accept things that might be dangerous, Dix says, then it would be your responsibility as a developer to make sure that the requests that might be dangerous came from people who knew what they were doing, and that you built in security.

Unlike the exchange of documents — spreadsheets and word processor files — between two people who can use public key infrastructure (PKI) or other measures to identify each other, distributed software components will communicate with one another without human intervention. In the new net services platforms such as Microsoft’s .Net, Novell’s One Net and Genuity’s Black Rocket, distributed software components will be everywhere, placing remote procedure calls (RPCs) to one another using the XML protocol. So why is the W3C close to deciding that security is not an issue, at least for them?

Should W3C Address Security Concerns?

The security debate began last May, when Ken MacLeod, an engineer of XML-RPC — a SOAP forerunner — published an article and posted a link to it in the W3C mailing list used by SOAP’s key engineers. While some rigorously developed applications may be thoroughly screened for security holes, MacLeod wrote, the vast majority of applications will never have security as a high priority. He went on to write that the syntax and content of RPCs are based on APIs, and that APIs are subject to frequent change. Every time an API is altered or amended, wrote MacLeod, security analysts would need to reassess the implications.

Many who began dismissing SOAP in the belief that it would be insecure, according to professional developer James Snell, author of a forthcoming SOAP book for O’Reilly, may have done so because it was originally marketed as a great way to do RPC.”

Speculation arose that SOAP could lead to a nightmare situation where one program could automatically hook deep into another program — and the owner would have no idea what had been done, and no way to prevent it.

There’s been a lot of concern it’s not a secure protocol because they didn’t define any security, says Snell. He explained, however, that security was never SOAP’s intention: It’s just an envelope for packaging data. In other words, you can’t blame an envelope for not being a safe. To reassure those who are still worried, Snell adds, Nothing in SOAP is automatic; just by using SOAP, your system doesn’t automatically open up.

Snell’s viewpoints are shared by many at W3C, including representatives of Xerox, who recently posted this:

Authentication, encryption and reliable delivery are already addressed at the level of protocols like HTTP and SMTP.

RPCs and the sessions that bind them are inherently complex, the Xerox engineers wrote, and any attempt by the XML Protocol to address these complexities would be redundant.

The XML Protocol is evolving into a way of using XML to encode data in a way that anybody can read it, no matter what operating system or language, according to Snell.

People should think more about the concept of interoperability than merely RPC. SOAP is more like a universal API. Without SOAP, you’re constantly writing specific APIs between applications — CORBA apps can speak only to CORBA apps, COM apps to COM apps. With SOAP, CORBA can natively interact with COM and vice versa. It’s an Internet standard way of communicating — no single company can get a lock in.

Securing Remote Procedure Calls

As the company best known worldwide for being able to get a lock in, Microsoft is recognized today as SOAP’s leading proponent. Microsoft promotes SOAP as a lightweight protocol for the exchange of both information and RPCs in a decentralized, distributed, networked environment.

The principle of RPCs dates back to Microsoft’s creation of the Component Object Model (COM), a way for small parts of programs (libraries) to be linked together as one program at the time the user runs the application, as opposed to the time the programmer compiles its source code. To move COM out of the confines of a single processor and over the Internet, Microsoft developed Distributed COM (DCOM), which let Windows applications make RPCs to other Windows apps.

Ironically, developers were originally attracted to SOAP, says Dix, because of some of the security nightmares they faced when trying to do DCOM over the Internet. It just was hard to get working, if you could do it at all. The security issues were just awful. CORBA had its own complexities as well. SOAP was written with the intent that people have to work inside of a corporate environment with a firewall, and need to be able to perform the sort of functionality. I know, the IT managers, as soon as you start talking about sending remote procedure calls as HTTP, get a chill down their spine.

SOAP’s dependence on HTTP and Internet port 80 as its primary transfer medium is a method that has been affectionately dubbed tunneling over firewalls. Although this sounds like a built-in measure for security breaches, Dix says, the technique actually relies upon most firewalls’ open acceptance of port 80 to get the message across. Because SOAP is XML and because it is transport independent, he says, it can be — and in the early examples, it has been — applied to sending messages as HTTP over port 80, and thereby circumventing some of the security issues with the firewall. Within the protocol and the specification, however, there are ways of identifying and using HTTP headers, and SOAP headers as well. As Dix explains, targeting SOAP messages for port 80 can enable content filters at the receiving end to scan for explicitly labeled SOAP messages — which could, if an administrator deemed it necessary, be blocked.

Here’s one example of a conceivably common SOAP session: A word processor could use HTTP to place a remote call in XML to a language translation application, requesting that its document be translated into a foreign language. The remote application would respond with an XML document containing the translation, in such a way that the end user would never be aware of the remote application.

How would these applications identify one another, and how is the exchanged data secured? Intentionally, SOAP by itself addresses neither question. Messages between applications are sent in the clear, meaning that anyone who intercepts the transmission and can read basic XML will have access to this information.

To protect yourself, advises Snell, you should at the very least encrypt the memo, as you would with confidential e-mail. In addition, you could send it over SSL, rather than insecure HTTP. Further, you could encrypt the SOAP envelope itself. A method for doing precisely that last item may have just arrived.

XKMS: A Solution In The Works?

The recent announcement by Microsoft, VeriSign and webMethods of a secure XML specification for digital signatures and encryption, called XML Key Management Specification (XKMS), promises to provide some security and peace of mind, at least for users. XKMS is a specification for managing public keys used to support digital signatures or encryption, or other applications of public keys, Verisign’s chief technology officer Warwick Ford tells us. So, it’s designed to work specifically alongside, and in conjunction with, the recent XML signature standard prepared jointly by IETF and W3C.

XKMS offers tools for digitally signing and encrypting documents shared between SOAP applications. So conceivably, a spreadsheet sent between computers could be both protected and authenticated, without engineers needing to amend SOAP itself. Although examples of XKMS for distributed object signing have yet to be investigated, Ford says, XKMS supports signing of XML objects…like business transactions. But it’s not limited to that. So, indeed, if you wanted to build some kind of software distribution system, which was itself an XML application, then you could use this mechanism for signing those objects.

Designing applications properly is the best way to minimize security holes, says Snell. It goes back to good application design — any application is insecure if you use it improperly. If a developer uses SOAP to write an application that accepts application code and then executes that code without first discovering where that code is coming from — ensuring that trusted relationship — that developer should be fired.

Dix suggests that security could be built into a SOAP enabled application by restricting the number of functions it exposes to the outside world — in developer parlance, by limiting its interface. Almost exclusively, says Dix, SOAP applications would not open up [broad] access to the components that exist on the server. Instead, he says, developers should adopt a very focused solution, one that was geared to exposing the functionality of one or a handful of components that you might have on the server.

As late as this week, proposals were being entertained by W3C to drop references to security measures in its upcoming XML Protocol draft, in favor of encouraging applications developers to build security into their own programs, and network administrators to monitor the communications channel. Whatever group provides the final answer to the XML Protocol security dilemma, it is now fair to assume that SOAP’s inner circle of engineers will not be part of it. Developers and security experts may have a rough job ahead.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Securing SOAP. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

I regret to announce the end of my efforts to mediate the Microsoft antitrust case

said Judge Posner in a statement released on Saturday, April 1st.

But it wasn’t.

After 4 months of discussions, any chance for a settlement between Microsoft and the US government seems to have ended fruitlessly. Many magazines, newspapers and web sites have been pointing out the fact that Judge Posner did not thank the states for their work in his brief statement. As a result, many believe that the states’ demands were largely responsible for the breakdown of the talks.

But the question remains as to what was the content of those talks. While most of the 20 drafts that were put together by the DOJ and Microsoft were kept a secret, word has leaked out regarding the content of some of those drafts. It includes:

• Uniform pricing structures that would disable Microsoft from offering preferential rebates and creating bundles for some partners.
• A provision forbidding Microsoft from striking exclusive contracts with other companies (for example, as it did with AOL, giving it an icon in Windows in exchange for packaging IE)
• Full disclosure of all the API to all Microsoft software, giving away what has long been believed to be a Microsoft advantage as it was supposedly allowing Microsoft internal programmers to develop software that better integrated with the Windows OS.
• Box manufacturers would be given source code to Windows and allowed to modify it at their will. They would be able to add and remove new features to it, such as, for example, changing the web browser. Microsoft would not be responsible for supporting those modified versions of windows.

Meanwhile, the 19 states involved in the negotiations would not go for anything less that a full break-up of Microsoft into several Baby Bills.

However, when you look at it closely, the proposal of the 19 states may be a bit shortsighted.

By creating a set of Baby Bills, we might end up with not one but several monopolies: One in the OS space (Windows has over 90% penetration in the consumer market), one in the application space (Office has more than 90% market share in the office suite market on both the PC and the Mac), and potentially in the Internet space (Internet Explorer has now supplanted Netscape Navigator as the browser with the largest market share).

However, while this case has become the computer equivalent of afternoon soap operas, with people tracking its every move, whatever happens this week when Judge Jackson presents a ruling that will most probably be unfavorable to Microsoft, may have little influence on the computer industry.

First of all, Microsoft will most probably appeal the ruling, going all the way to the supreme court and thus delaying its final impact for at least another year.

For starters, a lot of the battle is around Microsoft Windows and its larger implications in terms of helping Microsoft control the Internet. However, the landscape has dramatically changed since the DOJ and 20 states filed their case against Microsoft. The case was originally kicked off by Netscape as a salvo against Microsoft in the browser battle. Since then, Netscape has become a unit of the new behemoth in the Internet industry: America Online. AOL, which recently announced it would acquire Time-Warner, was playing both side of the fence. On the one hand, they are Netscape’s owners, and on the other, they are bundling IE as part of their client (of course, this allows them to also have an icon in Microsoft Windows, which has been in large part responsible for their success in customer growth).

At the time, Microsoft did have an inferior browser and was loosing in the marketplace. However, with the release of IE 4.0, and subsequent release of their 5.0 client, they started to gain market shares. Netscape was not happy about that. On the one hand, Microsoft was going faster than they were in terms of releasing new products. And, on the other hand, Microsoft was working on developing a version that would run better on four platforms: Windows, Macintosh, Solaris and HPUX. Meanwhile, Netscape was trying to support their browser on no less than 18 different platforms.

The interesting thing was that Netscape was complaining that Microsoft was going to compete with them. Yet, they were making it clear that they were out to:

replace Windows

and when they lost went to the DOJ to complain about Microsoft making their browser available for free even though they had adopted the same tactic long before Microsoft did.

Netscape’s failure was in the marketplace but since resentment of Microsoft’s success has always been a good thing to bet on, it played the legal card and got the proceeding started.

Meanwhile, Microsoft pushed more and more products out but failed to capture significant market shares in some critical Internet areas.

First, it attacked the online service business. AOL became the top online service, beating out Microsoft’s own MSN, even though MSN was bundled with Windows (this interesting little fact seems to show that bundling software with Windows is no guarantee that it will win out in the end.)

Realizing that it was loosing on that end, Microsoft decided to recast MSN as a portal but that decision was made too late and Yahoo! became the top search engine, followed by AOL’s own site.

On the server end, Microsoft is still trailing the free Apache server for the top position by a very large margin (Apache has a 60% market share, while Microsoft’s IIS has a 20% one). It’s efforts to cast its back-office suite of tools as the Internet suite of choice seems to have gone about as badly, with Microsoft trailing Oracle in the database space. Furthermore, some of Microsoft’s efforts to capitalize on their windows platform (remember ActiveX anyone?) have been received with less than enthusiasm by the development community… and let’s not forget Linux. While Microsoft was trying to make a big push against Solaris on the server end, the Linux crowd started going after the mid-size server market. As a result, Microsoft has not participated in the growth that other server-end operating systems have seen in the past few years. Linux has taken most of that growth away from Microsoft, representing a major threat to Windows NT. Linux has now become such a threat to Microsoft that the company is now breaking its own rule of never mentioning some other company’s software in presentations.

In new markets, Microsoft has also seen some very strong competition. While it has been moderately successful in the convergence space with WebTV (a company it bought for $420 million), the battle is just starting and AOL will soon be able to bring the power of its 22 million strong user base to the party, giving Microsoft’s some new headaches (WebTV has about 1 million users).

On the wireless end, Microsoft’s attempt to push Windows-CE against the PalmOS has been rebuffed and Palm computing has been very smart in licensing its operating system to cell phone vendors and other partners, creating a new platform Microsoft has to battle.

Last but not least, Microsoft is now trying to enter the gaming console market (with their X-box, promised for a Christmas 2001 release) and is thus entering another market in which it has little or no market shares.

When you look at this, you may say, well, why would anyone care about Microsoft’s dominance. It seems to be waning, right?

Well, yes and no. While the market is currently showing a company that’s fighting a war on multiple fronts, it’s also a company that has successfully managed to buy itself a seat at the telecom table (Microsoft now has investments in AT&T, Comcast, Nextel, and Qwest), as well as some other infrastructure plays (Akamai, Concetric Networks, Tut Systems).

What’s happened really, over the last few years, is that Microsoft stopped being a pure software company. Sure, it still sells software but its main business over the past few years has been as an investment firm. When you look at it this way, its dominance may somewhat make sense, much like Kleiner Perkins’ does.

As a result, I’m afraid a breakup could represent a major problem for our industry. For starters, look at the suggestion that’s been the most often floated: one company doing windows, another doing applications, and another doing Internet stuff. If you go with that breakup, you still have a dominant player in the OS market, a dominant player in the applications market (remember, this group would get MS Office) and a weak Internet company.

Most of Microsoft’s problems have stemmed from the fact that it has grown too big for its own good. For starters, it is now in so many markets that it is hard to figure out whether it really does wield as much power as it used to. Second, it is such a big organization that I seriously doubt programmers talk to each others as much as they should. However, if you were to break it up as more than three companies, you could get somewhere.

Let’s say we were to follow the logic of Microsoft as a VC firm. If we do so, it’s market cap is really not so huge, considering the number of companies it holds. It is, after all, a VC with over 200 companies in its portfolio.

Either way, the battle will go on and sometimes this week, a ruling will be issued but in the long run, I seriously doubt that anything that happens this week will have any lasting effect on the computer industry. Microsoft will win out or loose out in the marketplace, regardless of the outcome of this trial.

We’re living in too dynamic a marketplace for it all to make sense. In the end, however, we will all regret that case ever happening because it will set a precedent as to whether the government can get involved into IT affairs. It may be the only lasting effect we will all be forced to feel.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title MS-DOJ Talks Falter: So What?. You can follow him on twitter here or receive his weekly newsletter by subscribing here.

Webdriver lets Web developers access DirectX APIs through a combination of XML tags and JavaScript, giving developers a change to create online game and graphic-rich environment on the web. While it only runs on Windows platforms right now, this is a significant development because it allows any web developers to create software offerings that are now as graphically rich as any games you can find. Overall, this is great for games but I doubt it will catch fire unless they develop a version of it for other platforms.

However, if the WebDriver components become packaged with the most popular browsers, they could herald the arrival of the much talked about 3D environments we’ve read about in Gibson’s and Stephenson’s books. If you are running a windows machine, I’d recommend you download those drivers and try out their demos. They, not I, best showcase what this technology can do.
ivering a richer web experience if they get browser support. Until then, though, it will remain as exciting as VRML: great looking but still hanging out at the starting gate.

Tristan Louis is the founder and CEO of Keepskor and writes the influential tnl.net weblog, where this was initially posted under the title Wild Tangent: DirectX for the Web. You can follow him on twitter here or receive his weekly newsletter by subscribing here.