Metrics — Hard Metrics

Yes­ter­day, I looked at which parts of a com­pany can use met­rics. Today, I start delv­ing into the types of met­rics those dif­fer­ent groups can use. For the pur­pose of this on-going dis­cus­sion, I’ve divided met­rics into two cat­e­gories: Hard Met­rics and Soft Met­rics. The main dif­fer­ence betwen the two is that hard met­rics can eas­ily be mea­sured while soft met­rics are more amor­phous. One could also say that hard met­rics are more tra­di­tional, by nature, than soft met­rics. Let’s go into more details…

Rev­enue

One of the eas­i­est met­rics to mea­sure, in terms of assess­ing the value of a busi­ness, is the rev­enue lines and other items, such as profit and loss (also known as P and L), which one can cull from bal­ance sheets. While the raw num­bers, in and off them­selves have some value, it is impor­tant to look at dis­tri­b­u­tion. On the rev­enue side, dis­tri­b­u­tion could be spread across a num­ber of areas: for exam­ple, a site could receive some adver­tis­ing rev­enue (both through an inter­nal sales force or via an adver­tis­ing net­work like Google Adsense or Yahoo! Pub­lish­ing Net­work), derive rev­enue from paid sub­scrip­tions, receive dol­lars for syn­di­cat­ing con­tent to another source, or manke money from a num­ber of other areas. Some Web 2.0 com­pa­nies are deriv­ing rev­enues from the sale of prod­uct or ser­vices related to their free (adver­tis­ing sup­ported) offer­ings. Oth­ers might be look­ing at the sale of data they gather from the inter­ac­tion peo­ple have with their ser­vices. For exam­ple, I remem­ber see­ing a com­pany, a while back (and the name escapes me), which sold cor­po­ra­tion some track­ing mech­a­nism based on what peo­ple in the blo­gos­phere were writ­ing about the cor­po­rate client. This type of data min­ing will prob­a­bly become increas­ingly rel­e­vant and per­ni­cious in Web 2.x and I believe that it will become a major source of rev­enue for quite a few com­pa­nies. How­ever, those com­pa­nies will have to be care­ful in their offer­ing so they do not ostra­cize their audi­ence because another hard met­ric is…

Traf­fic

Since the onset of the com­mer­cial inter­net, traf­fic has been one of the hard met­rics relat­ing to inter­net busi­nesses. The rea­son such met­ric is of import is that, when look­ing at the web as a medium, the par­al­lel to other media make it eas­ier to grasp for mar­keters. As a result, dis­cus­sions of page views, eye­balls, vis­i­tors, and sites gen­er­ally exist around the time adver­tis­ers decide whether to make a pur­chase on a web site.

While the boom and bust of Web 1.0 has shown that pure page counts does not a busi­ness make, there is still some value to be derived from such num­bers. In Web 2.x, audi­ence par­tic­i­pa­tion is a mjaor fac­tor of suc­cess and it is the lock-in of cer­tain audi­ences that lends cer­tain Web 2.0 com­pa­nies some cre­dence. As a result, one should care­fully con­sider vis­i­tor counts and site num­bers when eval­u­at­ing the mar­ket poten­tials of a web business.

The vis­i­tor count (and its other met­ric the vis­i­tor growth data) can pro­vide some insight into the value of a par­tic­u­lar com­mu­nity. For exam­ple, when News Corp. bought MySpace, it not only bought a tool but also a com­mu­nity (and one that is grow­ing at a pretty quick pace). Or when Ebay bought Skype, it bought a sub­stan­tial num­ber of users along with some tools for VoIP. Whether those can be mon­e­tized remains to be seen but a con­cept of inte­gra­tion value will be attach­able to those metrics.

Also of import is the unique site count. The rea­son for this is that it gives more data as to vis­i­tor con­cen­tra­tion. If a com­mu­nity comes pri­mar­ily from sites in a par­tic­u­lar coun­try, one has to ask whether the value of that busi­ness exists only in that coun­try or whether it can be inter­na­tion­al­ized. Alter­nately, a wide dis­tri­b­u­tion and a slower growth in site count could mean that a web busi­ness has matured to the point of not being able to extend its reach any fur­ther (it could be that the busi­ness has reached all its poten­tial audi­ence already and there­fore will not grow much beyond its exist­ing posi­tion) As a result, an analy­sis of site dis­tri­b­u­tion can pro­vide some impor­tant value when try­ing to eval­u­ate a business.

Another impor­tant traf­fic met­ric, in the cur­rent world, is RSS sub­scribers. One could argue that sub­scribers are gen­er­ally more moti­vated cus­tomers in that they have taken a step towards get­ting to know a site a lot more. An RSS sub­scriber is some­one who is devel­op­ing a rela­tion­ship with a site, as opposed to more casual vis­i­tors of a site. There may be some added value in the level of engage­ment such a user would have. Some of the way one could poten­tially derive some value out of this type of user is in defin­ing focus group to get an idea as to what attracts cer­tain types of users to the site. Another rea­son to track RSS sub­scribers is that, accord­ing to sev­eral stud­ies (Pew, Yahoo), RSS sub­scribers tend to be in the early adopter camp and reach­ing such indi­vid­ual could be the key to the ini­tial launch of a suc­cess­ful product.

Out­put

Another use­ful met­ric to track is the out­put of a site. In terms of a blog, the out­put can be seen as num­ber of entries per day. In terms of Web 2.x busi­ness, the out­put could be seen as num­ber of page or remixes cre­ated through user inter­ac­tion. In a recent post on his blog, Fred Wil­son men­tioned the fol­low­ing met­rics track­ing he and his part­ner are doing on their investments:

In the case of Indeed, we like to watch the num­ber of searches per month.In the case of Deli­cious, we like to watch the num­ber of post­ings per month.

In the case of Tacoda, we like to watch the amount of behav­ioral data being cap­tured in the TAN network.

These can all be seen as a mea­sure of out­put for the dif­fer­ent com­pa­nies. Of course, the out­put is dif­fer­ent based on each com­pa­nies approach but it does pro­vide some info.

Trends

While Fred chides me for try­ing to assess some val­ues based on links, he men­tions how he and his part­ners are approach­ing the metric:What we do is use these num­bers to mon­i­tor our invest­ment the­sis and make sure the oppor­tu­nity is play­ing out the way we think it will. And we use them to find places where we need to work harder on the ser­vice to make it bet­ter. And we use them to show new hires, new part­ners, and new investors that the busi­ness is on a solid trajectory.These are actu­ally val­ues relat­ing to trends, which is the most impor­tant thing when track­ing hard metrics.

Growth pat­terns are really what most of this data should allow one to divine. One’s abil­ity to judge such pat­terns (and assess whether they are headed in the right or wrong direc­tion) is the dif­fer­ence that will make or break many a busi­ness. When look­ing at all this data, look­ing at it in a closed fash­ion (ie. a slice of data in time) does only do it jus­tice when try­ing to assert a base­line. How­ever, base­lines in and of them­selves are not that inter­est­ing. The real inter­est really lies in the trends to evolves over time against those baselines.

Engage­ment trends

I would assert (and will prob­a­bly receive many a flame for it) that the value of web 2.x busi­ness is going to lie partly in what I would call the “engage­ment trend line”, which I see as a value derived in part from sev­eral engage­ment met­rics. On a weblog, those met­rics could be bro­ken out as fol­lows: RSS sub­scribers, click through from those sub­scribers to the blog, com­ments posted to that blog, and links towards that blog. Yet to be defined, how­ever, is whether Metcalfe’s law of net­work effect actu­ally does have an impact on all of those.

Tomor­row, I will look into the soft met­rics, which are harder to mea­sure but, for now, I’d like to ensure that the dis­cus­sion goes on so please either post your com­ments in the dis­cus­sion area or com­ment on this on your own blog.