September 5, 2009
Elko I: The Life, Death, Life, Death, Life, Death, and Resurrection of The Elko Session Server
Preface: This is the first of three posts telling the story of Elko, a new old new piece of web server infrastructure that I’m releasing this week as open source software. Part I, this post, is a bit of personal history. There’s a companion technical story concerning the whats and the hows and the whys that’s fairly profound on its own. But though the technical story is intimately connected to this one, it’s got a very different narrative. The technical backstory is the subject of Part II, which I’ll be posting tomorrow, and a more detailed technical overview will presented in Part III, which will be up the day after tomorrow.
There is a significant piece of very cool software that, through circumstances largely beyond my control, I have come to write, from the ground up, three times. Consequently, it’s pretty mature even though it’s not. I will share the tale with you now.
Software incarnation #1 was done at the late lamented Communities.com (no relation to whoever is sitting on that domain name today). It was a system called Passport that I created in 1999 and 2000 by transmuting the server code for The Palace into a general purpose server for stateful applications. It was actually the most financially successful thing Communities.com ever produced (used as the basis for a project we did for Turner Broadcasting’s Cartoon Network, called CartoonOrbit), but wasn’t enough to save the company. Communities.com crashed and burned and went off to the big IPO in the sky, ultimately filing Chapter 7 on March 5, 2001 (my 42nd birthday, Happy Birthday to me).
Unfortunately, there was not much we could salvage from the wreckage. The IP assets of Communities.com ended up being taken over by one of our key investors as partial payment for a secured debt we had to them. However, they were neither prepared to do anything with the stuff themselves nor to talk business with anybody else who might want to make a deal for it. (A perfect fit to Carlo Cipolla’s classic definition of stupidity: A stupid person is a person who causes losses to another person or to a group of persons while himself deriving no gain and even possibly incurring losses.)
However, given our modest success with Turner Broadcasting, the survivors of Communities.com felt there was potentially a bankable product here. Alas, as just described, we no longer had rights to the code. Consequently, this lead to software incarnation #2, which I wrote from scratch over the late spring and summer of 2001, based on some of the ideas we had in Passport and a few more we had come up with in the meantime. This was 100% original Java code (the Passport server had been written in C), that became the basis for the company that Doug Crockford and I started in 2001, State Software (which Randy also joined a few months later). The server and its accompanying client components became the State Application Framework, SAF. Whereas Passport had required the installation of a special browser plugin to do magic network communications things, SAF was entirely HTTP-capable and the client was Javascript running in an unmodified browser. This was an early instance of the kind of thing that is nowadays called Ajax, though of course nobody knew this at the time. (Basically, Crock and I can legitimately claim to be inventors of Ajax, along with people at probably 30 other companies at the same time who also invented it, though Jesse James Garrett gets the credit (and, let me hasten to add, in my opinion deserves it) because he managed to take the idea mainstream and gave it a cool name that stuck — kind of the way Columbus discovered America: lots of other people had discovered America before him, but after Columbus nobody else could discover it any more.) As it turns out, being alienated from the fruits of our labors at Communities.com proved to be a blessing in disguise, since, in addition to being unencumbered from an intellectual property standpoint, the new server was vastly more capable, more flexible, more scalable, and phenomenally more robust.
State Software lasted about a year and a half before crashing at the end of the runway with insufficient fuel to take off. We were a couple of years ahead of the market and our financial backer found himself unable to sustain things. In this case, the guy was a money manager for another guy with a pile of money, but he turned out to be an amateur as a venture capitalist. Basically, he bet almost all the money guy’s money on a bunch of first round investments, embezzled the remainder, and left nothing in the tank to provide followup support for the few companies that had proved promising after the first year. State Software (and the rest of the investor’s portfolio) collapsed into a filing cabinet in some lawyer’s office somewhere. Feckless VC guy eventually went to jail. Once again, cool code that I had written vanished from my grasp.
Fastforward a few years and a number of colorful adventures later, and I find myself at Yahoo! along with Randy and Crock. Crock and I were pondering various Ajax-y things then being done at Yahoo!, waxing nostalgic, and thinking how cool it would be to have access to the State Application Framework for some of the more advanced community services we were then contemplating. After some tentative and unhelpful inquiries with the current owners of the assets about what it would cost to reacquire the rights (answer: for you, 1.5*$N, where $N is defined as the maximum you’d be willing to pay, regardless of who you are, a flagrant violation of Morningstar’s First Rule of Business if ever there was one), we got to thinking. Passport, software incarnation #1, had taken roughly two years. SAF, software incarnation #2, had taken about four months (well, plus lots of follow-on tweaking and enhancing over the life of the company, but basically four months for the central code). What if we just did it over again from scratch? We had our memories of what we’d done before, and we even had the API docs for SAF, which had been widely published. Sure, it’d be annoying to have to go through the bother, but how hard could it be, really? Well, for one thing, I didn’t have a mandate from Yahoo! to spend time on this, so it would have to be a personal time project. On the other hand, if it worked out I might be able to leverage it into some things at Yahoo! that would be way more fun than what I was working on at the time. So over the 2006 Christmas break I set to work on a project I code named Elko. (Digression: WorldsAway was initially named Reno; EC Habitats, AKA Microcosm, was initially named Lodi; the working name for the thing I’m currently doing at WeMade is Yuma. I’ve decided that my personal project namespace is seedy-western-towns-with-four-letter-two-syllable-names. I figure that with Orem, Ojai, and Moab I’ve probably got enough names to last for several years before I have to crack open an atlas.)
After two weeks of coding in what I can only describe as the most transcendent state of flow I’ve ever experienced, I produced software incarnation #3. Like the four month initial version at State, it was not the whole system (and I’d end up spending a fair bit of my free time over the next year doing the tweaking-and-enhancing thing), but it was the essential central piece. Crock and I came up with the idea of branding it the Yahoo Ajax Server (YAS). He did some client-side stuff to show it off and we set about trying to evangelize it to the company. There was some abstract interest, but planning lead times were such that projects were typically committed to a particular technical course long before hitting the problems for which this technology was the solution, at which point it was too late to reopen fundamental architectural questions. We found we needed to be talking to people at a much earlier stage in the product pipeline, and the months dragged on as we sought the ears of the right people. Then, in February 2008, a wave of cost cutting swept me (and Randy) out the door, an event that regular (such as it is) readers of these pages know all about. For the third time, I had created this piece of cool technology and then its shot at the big time had fallen to the vicissitudes of the business world before it made it into the hands of anybody who could actually do something productive with it.
Then one day a few weeks ago we were sitting around at WeMade thinking about real-time chat solutions for the community system we were developing, and I remember I’ve got this Elko thing laying around in my projects folder. We could just use that. And anyway, it’s good stuff, so I’ve decided to clean it up and make it available to everybody.
So, here it is. I’m releasing it to the world under the MIT license, in hopes it may prove useful.