Habitat Chronicles

Fujitsu Habitat footage on YouTube

Tomoko Kojima (aka Oracle Layza) found some video of very early Fujitsu Habitat interaction (on the FM-Towns) on YouTube:

and

.

I’m most struck by the whiplash discrepancy between the music and the action in the first clip. That and the way the Internet continues to cough up nuggets of wonder from the distant past.

Thanks Tomoko!

History | Posted by Chip at 10:52 AM | 1 Comment

Elko I: The Life, Death, Life, Death, Life, Death, and Resurrection of The Elko Session Server

Preface: This is the first of three posts telling the story of Elko, a new old new piece of web server infrastructure that I’m releasing this week as open source software. Part I, this post, is a bit of personal history. There’s a companion technical story concerning the whats and the hows and the whys that’s fairly profound on its own. But though the technical story is intimately connected to this one, it’s got a very different narrative. The technical backstory is the subject of Part II, which I’ll be posting tomorrow, and a more detailed technical overview will presented in Part III, which will be up the day after tomorrow.

There is a significant piece of very cool software that, through circumstances largely beyond my control, I have come to write, from the ground up, three times. Consequently, it’s pretty mature even though it’s not. I will share the tale with you now.

Software incarnation #1 was done at the late lamented Communities.com (no relation to whoever is sitting on that domain name today). It was a system called Passport that I created in 1999 and 2000 by transmuting the server code for The Palace into a general purpose server for stateful applications. It was actually the most financially successful thing Communities.com ever produced (used as the basis for a project we did for Turner Broadcasting’s Cartoon Network, called CartoonOrbit), but wasn’t enough to save the company. Communities.com crashed and burned and went off to the big IPO in the sky, ultimately filing Chapter 7 on March 5, 2001 (my 42nd birthday, Happy Birthday to me).

Unfortunately, there was not much we could salvage from the wreckage. The IP assets of Communities.com ended up being taken over by one of our key investors as partial payment for a secured debt we had to them. However, they were neither prepared to do anything with the stuff themselves nor to talk business with anybody else who might want to make a deal for it. (A perfect fit to Carlo Cipolla’s classic definition of stupidity: A stupid person is a person who causes losses to another person or to a group of persons while himself deriving no gain and even possibly incurring losses.)

However, given our modest success with Turner Broadcasting, the survivors of Communities.com felt there was potentially a bankable product here. Alas, as just described, we no longer had rights to the code. Consequently, this lead to software incarnation #2, which I wrote from scratch over the late spring and summer of 2001, based on some of the ideas we had in Passport and a few more we had come up with in the meantime. This was 100% original Java code (the Passport server had been written in C), that became the basis for the company that Doug Crockford and I started in 2001, State Software (which Randy also joined a few months later). The server and its accompanying client components became the State Application Framework, SAF. Whereas Passport had required the installation of a special browser plugin to do magic network communications things, SAF was entirely HTTP-capable and the client was Javascript running in an unmodified browser. This was an early instance of the kind of thing that is nowadays called Ajax, though of course nobody knew this at the time. (Basically, Crock and I can legitimately claim to be inventors of Ajax, along with people at probably 30 other companies at the same time who also invented it, though Jesse James Garrett gets the credit (and, let me hasten to add, in my opinion deserves it) because he managed to take the idea mainstream and gave it a cool name that stuck — kind of the way Columbus discovered America: lots of other people had discovered America before him, but after Columbus nobody else could discover it any more.) As it turns out, being alienated from the fruits of our labors at Communities.com proved to be a blessing in disguise, since, in addition to being unencumbered from an intellectual property standpoint, the new server was vastly more capable, more flexible, more scalable, and phenomenally more robust.

State Software lasted about a year and a half before crashing at the end of the runway with insufficient fuel to take off. We were a couple of years ahead of the market and our financial backer found himself unable to sustain things. In this case, the guy was a money manager for another guy with a pile of money, but he turned out to be an amateur as a venture capitalist. Basically, he bet almost all the money guy’s money on a bunch of first round investments, embezzled the remainder, and left nothing in the tank to provide followup support for the few companies that had proved promising after the first year. State Software (and the rest of the investor’s portfolio) collapsed into a filing cabinet in some lawyer’s office somewhere. Feckless VC guy eventually went to jail. Once again, cool code that I had written vanished from my grasp.

Fastforward a few years and a number of colorful adventures later, and I find myself at Yahoo! along with Randy and Crock. Crock and I were pondering various Ajax-y things then being done at Yahoo!, waxing nostalgic, and thinking how cool it would be to have access to the State Application Framework for some of the more advanced community services we were then contemplating. After some tentative and unhelpful inquiries with the current owners of the assets about what it would cost to reacquire the rights (answer: for you, 1.5*$N, where $N is defined as the maximum you’d be willing to pay, regardless of who you are, a flagrant violation of Morningstar’s First Rule of Business if ever there was one), we got to thinking. Passport, software incarnation #1, had taken roughly two years. SAF, software incarnation #2, had taken about four months (well, plus lots of follow-on tweaking and enhancing over the life of the company, but basically four months for the central code). What if we just did it over again from scratch? We had our memories of what we’d done before, and we even had the API docs for SAF, which had been widely published. Sure, it’d be annoying to have to go through the bother, but how hard could it be, really? Well, for one thing, I didn’t have a mandate from Yahoo! to spend time on this, so it would have to be a personal time project. On the other hand, if it worked out I might be able to leverage it into some things at Yahoo! that would be way more fun than what I was working on at the time. So over the 2006 Christmas break I set to work on a project I code named Elko. (Digression: WorldsAway was initially named Reno; EC Habitats, AKA Microcosm, was initially named Lodi; the working name for the thing I’m currently doing at WeMade is Yuma. I’ve decided that my personal project namespace is seedy-western-towns-with-four-letter-two-syllable-names. I figure that with Orem, Ojai, and Moab I’ve probably got enough names to last for several years before I have to crack open an atlas.)

After two weeks of coding in what I can only describe as the most transcendent state of flow I’ve ever experienced, I produced software incarnation #3. Like the four month initial version at State, it was not the whole system (and I’d end up spending a fair bit of my free time over the next year doing the tweaking-and-enhancing thing), but it was the essential central piece. Crock and I came up with the idea of branding it the Yahoo Ajax Server (YAS). He did some client-side stuff to show it off and we set about trying to evangelize it to the company. There was some abstract interest, but planning lead times were such that projects were typically committed to a particular technical course long before hitting the problems for which this technology was the solution, at which point it was too late to reopen fundamental architectural questions. We found we needed to be talking to people at a much earlier stage in the product pipeline, and the months dragged on as we sought the ears of the right people. Then, in February 2008, a wave of cost cutting swept me (and Randy) out the door, an event that regular (such as it is) readers of these pages know all about. For the third time, I had created this piece of cool technology and then its shot at the big time had fallen to the vicissitudes of the business world before it made it into the hands of anybody who could actually do something productive with it.

Then one day a few weeks ago we were sitting around at WeMade thinking about real-time chat solutions for the community system we were developing, and I remember I’ve got this Elko thing laying around in my projects folder. We could just use that. And anyway, it’s good stuff, so I’ve decided to clean it up and make it available to everybody.

So, here it is. I’m releasing it to the world under the MIT license, in hopes it may prove useful.

History | Posted by Chip at 5:20 PM | No Comments

FACT CHECK: Lucasfilm's Habitat in Rogue Leaders

Recently, GameSetWatch published an excerpt from Rogue Leaders about Lucasfilm’s Habitat which includes several new images and wonderful details.

Unfortunately, it also contains several factual and categorical errors that need to be corrected in the public record since this book’s account has already been used to incorrectly update Habitat’s Wikipedia page.

This article will block-quote the relevant sections of the book, followed by factual corrections marked as FACT CHECK: or commentary marked with either Chip: or Randy: as appropriate.

Q-Link, as it was known, undercut that price to around $3.60 an hour by renting out spare, unused server space during low-usage times.

FACT CHECK: The underused, and therefore discounted, resource was not servers, but off-peak packet-switching network bandwidth.

Through this partnership a deal was hatched to produce an online game, with Lucasfilm Games creating the front-end game — Habitat — on the Commodore 64, and Q-Link producing the back-end, server-side software.

FACT CHECK: Lucasfilm also developed a large portion of the backend. Q-Link, lead by Janet Hunter, did the stuff that had to interface with their system, but Lucasfilm did the game-specific stuff.

Designer Noah Falstein had been working with one of the team engineers, Chip Morningstar, on the game concept.

Chip: That’s a little backwards. The original concept emerged from a collaboration between Noah and me, but the design itself was mine. We were all peers with the same title, “Designer/Programmer”, with an equal emphasis on concept and implementation.

Randy: See Chip’s post on the beginnings of Habitat for a detailed account those early days.

The game debuted internally at Lucasfilm Games at a company meeting in early 1988.

…

It looked like Habitat was a huge hit-in-the-making, and so in the fall of 1988 the beta was taken to a New York nightclub for a launch party as Lucasfilm Games and Q-Link prepared to revolutionize gaming.

FACT CHECK: Summer and Fall 1986, after the game had first been shown to selected industry and press people at the Chicago CES in June.

Randy: Watch the Habitat Promotional Video and it’s copyright date for verification.

Essentially, if 500 users were so committed to playing Habitat that they remained online long enough to eat up 1 percent of the network’s entire system bandwidth, a full-run production that could attract Rabbit Jack’s Casino numbers could boost that bandwidth number to 30 percent. “The way the system was built, the server software wasn’t capable of hosting that population while still being successful,” recalls Arnold.

Ultimately, these business challenges caused Habitat to be cancelled after the launch party, but before it had gone into full production and reached retail shelves. It would simply be too popular, and the necessary server fix would be too expensive to make the project viable. And so this massively original, inventive, and cutting-edge project was shelved for U.S. release.

From a business perspective, however, Habitat wasn’t a failure. The game was licensed to Fujitsu for use on its FM Towns PC-like platform, and the successor to Habitat was recast (with several of the original planned features now cut) as Corpe Caribe, described as an online Club Med, where it enjoyed some success.

FACT CHECK: The shipped product was Club Caribe, not “Corpe Caribe”. :: sigh ::

Chip: While there were some performance tuning issues that needed to be addressed, the cost of operations was never really the issue. Statements about performance considerations were a face saving way of covering for the what Q-Link perceived as the real problem, which was marketing risk. Basically, the product was so weird and out of the mainstream that they didn’t think they knew how to sell it. In particular, for some reason they felt that people would be put off by the fantasy and science fiction elements. We argued that this defied everything we knew from the history of computer games, but they believed their typical user was far more conventional and unimaginative than the typical game purchaser.

FACT CHECK: Club Caribe was Habitat and it was released commercially by Q-Link. It opened in January 1988 with the name change and a different marketing spin. Literally the only difference between the original Habitat client software and the Club Caribe software as shipped was the title screen image.

Chip: Basically, Q-Link reworked the world database to remove any of the objects that had any kind of fantasy or science fictional flavor. The idea was to make the world seem more ordinary, pitching it as a virtual resort. Notably, they didn’t use any avatar heads that were non-human.

Chip: Over the course of the first six months of operation, as they grew more comfortable with their users, all these pointless restrictions were eventually abandoned.

The licensing to Fujitsu for the FM Towns happened a couple years later.

Chip Morningstar and Randy Farmer, the two programming gurus who had built the system infrastructure

Chip and Randy: That’s a slight mischaracterization of our role. While we certainly programmed it, we think it’s more noteworthy that Chip designed the whole thing, and Randy ran the world. Both the design itself and our operational experience with it are arguably quite a bit more important to the historical significance of Habitat than was its implementation.

Randy: Well, except that we managed to get a virtual world client shoe-horned into a 1-megahertz, 300-baud, 64k-memory computer with a 165k floppy disk! Certainly not a fact of wide ranging repercussions, but still pretty damn impressive.

History | Posted by Randy at 2:45 PM | 5 Comments

Interview about building Lucasfilm’s Habitat (LONG!)

CircuitGirl/Jeri Ellsworth posted a video interview with Chip and me back in November about the making of Lucasfilm’s Habitat/Club Caribe for the QLink service during the mid-late 1980s.

Warning:– it’s and hour long, and the audio goes a little wonky a bit through. Oh yeah, and I mispeak about my first project with Lucasfilm Games – it was Koronis Rift, not Rescue on Fractalus – I was running on 5 hours sleep over two days at the time…

History | Posted by Randy at 2:35 PM | 3 Comments

Mud/Moo/Virtual World Pioneers Panel

Richard Bartle, Randy Farmer, Pavel Curtis

Raph Koster, a first class pioneer himself, posted a set of pictures of Richard Bartle, Pavel Curtis, Brian Green, himself, and myself taken just before the pioneer’s panel at Living Games Worlds. First time we were all in one place. I knew them all, but Bartle and Curtis had never met!

We all missed having the real brains behind Lucasfilm’s Habitat there: Chip Morningstar. Hope you’re feeling well buddy, and I hope you like how I covered our spot. :-)

History | Posted by Randy at 6:40 PM | 1 Comment