TNL.net moves off Windows

Over the past year, it has seemed like TNL.net had gone quiet. Few updates were made to the site and fewer newslet­ters were being pub­lished. Behind the scene, how­ever, I was busy rebuild­ing the site from the ground up. The new TNL.net was relaunched in late Jan­u­ary 2003, about a month behind the sched­ule I had orig­i­nally set. In this newslet­ter, I will explain why I relaunched it and how I went about it.

The Gen­e­sis

Since 1995, TNL.net was run­ning on Microsoft Win­dows. Ini­tially, the site was run­ning on Win­dows NT, using Microsoft IIS to serve pages on the web. Prior to that, TNL.net had been run­ning on a shared Linux box but none of the admin­is­tra­tion of that server was in my con­trol. When it moved to its own envi­ron­ment, I looked at a num­ber of options that would allow me to spend time devel­op­ing the site, not nec­es­sar­ily admin­is­ter­ing the box. In late 2000, the site was upgraded to Win­dows 2000 and con­tin­ued to run on this until recently. So why did I decided to change things?

Unhappy with Microsoft

The world in 1995 was very dif­fer­ent from today’s world. In those days, admin­is­tra­tion of unix sys­tem required a set of spe­cial­ized skills and admin­is­tra­tors had to design their own pro­grams to auto­mate a lot of work. Over the years, more prod­ucts started com­ing out of the open source com­mu­nity that would sim­plify the way one man­ages a Linux box.

Step 1: Mov­ing Up, Mov­ing Out

In par­al­lel, Microsoft started going fur­ther and fur­ther upstream in search of new rev­enues for win­dows. Hav­ing suc­cess­fully con­quered the desk­top, they first set their sight on small busi­ness. With the intro­duc­tion of Win­dows 2000, they started upping the price of inter­net servers, requir­ing a sep­a­rate set of licenses for the inter­net ser­vices as an add-on to the oper­at­ing sys­tem costs.

As TNL.net grew more and more pop­u­lar, it became quickly appar­ent that the small MS-Access data­base that was pow­er­ing the server needed to be upgraded.

Step 2: Patch, Patch, Patch

After buy­ing a license to SQL server, I learned that I had to start wor­ry­ing about another set of patches on a weekly basis. As new secu­rity patches were released weekly for win­dows 2000, IIS, SQL server, I found myself spend­ing more time doing basic admin­is­tra­tion of the box and less and less time doing devel­op­ment on it. With a lim­ited time resource, this became a first point of con­cern and pushed me an area where I started ques­tion­ning whether I wanted to stick with the plat­form for the long term.

Step 3: Price Hike

After care­ful study of the win­dows roadmap, it became appar­ent that Microsoft was no longer inter­ested in small and medium size sites and was get­ting more and more tar­get­ted towards the enter­prise. The intro­duc­tion of .net was clearly aimed at the enter­prise mar­ket. The sub­se­quent announce­ment of changes to their licens­ing struc­ture showed that they would dras­ti­cally increase prices and force you into an upgrade cycle, whether you liked it or not. Microsoft was essen­tially telling me that if I wanted to play long term, I would not only be forced to pay more but also be forced into buy­ing soft­ware on a sub­scrip­tion basis, whether I liked it or not. This would not do. I was already an eager Microsoft user and, hav­ing paid for all my licenses, had already laid out a sub­stan­tial amount of cash into stay­ing with Microsoft.

Step 4: Locked up

To look at a change in plat­form because of a new pric­ing model was an impor­tant step but was it really what I wanted? I decided to approach a few peo­ple I knew in the com­mu­nity and within Microsoft to see what the gen­eral view was. Maybe I was over-reacting.

Ser­vice Pack 3 showed me that I was not. With it, Microsoft essen­tially was telling me “open your box to us and we will take care of every­thing”. Every­thing, that is, that Microsoft deemed OK. Which essen­tially meant that I would let Microsoft decide what could and couldn’t be on my box. For purely his­tor­i­cal pur­pose (as I expect this story, like all the oth­ers on TNL.net, to stay on the web for many years to come, you have to real­ize that all this is hap­pen­ing against the back­drop of Microsoft being inves­ti­gated for antitrust violation)

Step 5: Look­ing to the other side

One of the things that allowed the Inter­net to grow as quickly as it did over the past decades is the fact that most peo­ple are will­ing to share. How­ever, in the win­dows world, the logic is that what you cre­ate is for your own and you charge for it. This left me in the odd posi­tion of being one of the very few open-source win­dows devel­op­ers in the world. If I had a cool piece of ASP writ­ten, you just had to ask and I would send you the code.

Mean­while, the open source com­mu­nity was pro­duc­ing prod­ucts that were grow­ing more sta­ble by the day and at a frac­tion of the costs offered by Microsoft. Most of the prod­ucts were run­ning on Linux, using Apache as their web server. By 2001–2002, they had grown mature enough to being con­sid­ered enter­prise grade.

Check­ing out the alternative

Once I decided that I would move to an open source plat­form, I had to inves­ti­gate the alter­na­tive. The ques­tion became what oper­at­ing sys­tem, what web server, whether to use another appli­ca­tion server, what lan­guage to use for devel­op­ing my back-end, whether to use a data­base (and if so, which one).

Choos­ing the Oper­at­ing System

Should I go with Linux or BSD or some­thing else? If I went with Linux, which dis­tri­b­u­tion should I fol­low. In the end, I decided to set­tle on Linux because of the wide level of sup­port if had not only on the open source mar­ket but also in the com­mer­cial one. With back­ers like IBM, Ora­cle, and even (begrudg­ingly) Sun Microsys­tems, I feel that Linux is here to stay and will con­tinue to grow not only as a hob­by­ist OS but also, increas­ingly, as one of the foun­da­tion tools for web busi­nesses of the future.

How­ever, look­ing at the dif­fer­ent Linux dis­tri­b­u­tion was a much more com­pli­cated mat­ter. I wanted some­thing that would require lit­tle day to day admin­is­tra­tion and could be eas­ily and quickly patched. At the same time, I had to ensure that the oper­at­ing sys­tem would run on the new TNL.net box, a dual 1.3Ghz Pen­tium con­fig­u­ra­tion with a SCSI RAID array. Unfor­tu­nately, I learned one of the tough lessons of the Linux world very quickly: Hard­ware dri­vers in the Linux world do not always get upgraded at the same speed as the rest of the Oper­at­ing Sys­tem does. In the case of my box, I dis­cov­ered that, while there is a new ker­nel patch com­ing out every few weeks, it is impos­si­ble to keep a machine up to the lat­est ker­nel if you have a pro­pri­etary set of RAID dri­vers. This is a shame on the part of the dri­ver mak­ers who could either pro­duce source code for the dri­vers and let the com­mu­nity update them or keep up with the demand. Attempt­ing to do a ker­nel upgrade proved to be a fruit­less effort as my options were either upgrade and lose the RAID array con­fig­u­ra­tion or wait until the ven­dor releases new RAID drivers.

In the end, I decided to pick up Red­Hat Linux 7.3 because my RAID ven­dor sup­ported it and because it offered supe­rior auto-updating fea­tures through the Red­Hat Net­work. Some mem­bers of the open source com­mu­nity will groan as Red­Hat is often seen as the Microsoft of the Linux World but I feel that it is cur­rently the eas­i­est point of entry for an ex-Microsoft user like myself.

Only one choice on web­server: Apache

Hav­ing made the deci­sion as to what Oper­at­ing Sys­tem to use, I turned my atten­tion to the web server. This deci­sion was a much eas­ier one to take as the web server from the Apache Foun­da­tion as become the de-facto stan­dard for any­one who is seri­ous about deliv­er­ing web con­tent on an open source plat­form. I did not go very far in my research on alter­na­tive web servers. The only deci­sion point on the web server end was whether to use Apache 1.3 or Apache 2.0. Influ­enc­ing this deci­sion was the mod­ule archi­tec­ture that Apache offers. Unfor­tu­nately, at the cur­rent time, Apache 2.0 is too new to have as exten­sive a set of mod­ules as Apache 1.3. As a result, I decided to act con­ser­v­a­tively and use the 1.3 version.

Back­end: What to use?

Com­ing from the Microsoft world, I have had exten­sive expe­ri­ence with ASP and could have kept using it thanks to a help­ful mod­ule devel­oped to run the Microsoft tech­nol­ogy on Linux. How­ever, I had already made the deci­sion that I would rewrite sub­stan­tial parts of my code so I decided to com­pletely aban­don ASP as part of the migration.

My choices were there­fore perl, PHP, Java, and Python. Each of the lan­guages was equally good for what I wanted to do.

Perl, which I have used for years (the orig­i­nal TNL.net was run­ning in Perl as Microsoft had not put out any offer­ing at the time) would have been my first choice if I had wanted to do a quick and dirty migra­tion. How­ever, Perl had got­ten old and was not designed per say as a web script­ing language.

Python looked very inter­est­ing but the syn­tax seemed a lit­tle too arcane for my taste.

Java looked inter­est­ing but, hav­ing made the deci­sion to leave the Microsoft plat­form because I did not want to become beholden to a sin­gle com­pany, would I end up mak­ing the same mis­take by going the Java route and become increas­ingly beholden to Sun Microsys­tems. That par­tic­u­lar fear kept me from going the Java route.

PHP seemed to have man­aged the proper bal­ance between extens­abil­ity and ease of use. It seemed like a new lan­guage devel­oped specif­i­cally for web use and I had not played with it. The big ques­tion on my mind, though, was whether it could sup­port large amounts of traffic.

At the time of the migra­tion, TNL.net was already received upwards of 3 mil­lion page views a month and grow­ing slowly towards 4 mil­lion. As if some­one had read my mind, Michael Rad­win made a pre­sen­ta­tion high­light­ing how Yahoo had come to the deci­sion of using PHP. Read­ing their analy­sis of the dif­fer­ent lan­guages, I felt that some­one else had gone through the same thought process I did. While they did not include Python (why, I do not know). How­ever, their con­clu­sions were that PHP was solid enough to run large por­tions of the Yahoo site. If that was the case, there was now no doubt in my mind as to which script­ing lan­guage I would use.

Choos­ing a Data­base Package

Once I chose PHP as my devel­op­ment lan­guage of choice, the next step was in assess­ing which data­base I should use. After care­fuly con­sid­er­a­tion, I decided to use the mySQL pack­age largely because of its tight inte­gra­tion with PHP. Fur­ther­more, inves­ti­ga­tion showed that this is a rel­a­tively stan­dard con­fig­u­ra­tion known as LAMP and is being used by a num­ber of peo­ple all over the net. More pow­er­ful options are offered by other open source data­base pack­ages like Post­GresSQL but I came to real­ize that for most things, MySQL is the best pack­age for me because many of the fea­tures offered by other pack­age are on the list of things I do not plan to use at any time in the fore­see­able future (12 months).

Con­clu­sion

Hav­ing moved to an open source pack­age was much eas­ier than I had ini­tally expected. With a com­plete rewrite of the back-end TNL.net code, I was able to clean things up and make sure that the site was opti­mized to sup­port the traf­fic it’s been get­ting to date and still have some room to breathe in terms of sup­port­ing more traf­fic. In the process, I made a deci­sion to get rid of some old things like the DMOZ imple­men­ta­tion I had and intro­duce new ones like the TNL.net weblog, which now pro­vides some con­tent on a more reg­u­lar basis (at least 5 times a week). I have already seen some div­i­dend in my join­ing the open source com­mu­nity, as some of the code imple­mented by other peo­ple served as a blue­print for the new TNL.net. All and all, count me on the list of peo­ple who have made the switch from win­dows to Linux. I don’t think I will look back any time soon.