Catastrophic Prototyping and Other Stories

The following essay was originally published in Tracy Fullerton’s Game Design Workshop (2008). I had given talks in the past on prototyping, but this was the first essay I wrote on the topic, and I think it holds up. If you’re interested in the Advanced Prototyping talk Chris Hecker and I gave at GDC’06, check out the old site, and scroll down a bit.

My hard drive was full of failures. Twelve years after learning to program, I looked back on all my software: none of it was finished, and what was, wasn’t ambitious enough. The projects that started out ambitiously always seemed to fall back to earth, like failed rockets lacking the power to propel their own weight into orbit. Sure, there were interesting ideas in there, lots of wacky toys, and I had even attempted a few large projects, but none of them ever came together like the cool games and software I had always admired.

Sure, I had become a pretty good programmer, and learned to make cool stuff, but clearly none of it would ever amount to anything. I just didn’t have what it took.

I went to graduate school at Georgia Tech, and read some Chris Crawford. I learned that he had the same problem. But he didn’t think of it as failure. For him, this was an organic part of the development process. The failures filling his hard drive were actually “prototypes” that helped him decide which ideas were worth pursuing. For each good idea, there were a large number of stupid ones that didn’t work out. Failing, for this successful designer, was a way to find the good ideas. The revelation hit me like a ton of bricks. Maybe I had a chance after all.

Ken Perlin came to Georgia Tech, and gave a talk on his work with emotional software actors. His work blew my mind. He had an infinite series of cool little toys, which he considered sketches or studies. Master artists like Escher or Van Gogh don’t just sit down and crank out a finished piece. Artists create numerous sketches and studies before they undertake finished paintings, let alone masterpieces. Ken’s larger demos clearly built on top of what he had learned in previous ones. It all formed one long line of inquiry and research. In Ken’s world, my failures, which I was now calling prototypes, are like an artist’s studies, a necessary part of any major undertaking.

All of my software failures, which I was now thinking about as prototypes, sketches, and studies, had taught me a thing or two about design and programming. If you want to learn to draw, you have to make a ton of bad drawings first. The difference between practice and failure is simply a matter of attitude. One thing led to another, and my experience, plus being in the right place at the right time, led to an internship with Will Wright. He was working on a new game at the time, code named Spore, and had a small team working on prototypes.

I went to Walnut Creek for the summer, joined the impossibly small Spore team, and it all finally started to click into place. Maxis was in the thick of The Sims Online, and the other intern and I were placed in the hallway outside of Will’s office, next to the Elvis shrine, on folding tables. Under my “desk” was one of  Will’s old Macs, a fancy machine from the mid-90’s, which, to my delight, we hooked up. Spending the summer at Maxis was like going to Santa’s workshop at the north pole, and finding out how the elves made the toys.

The old Mac was like a treasure cave, a historical archive of blueprints, prototypes, projects, and concepts. I could study the source code to some of my favorite games, like SimAnt, SimCity, and SimCity 2000. But that wasn’t even the best part. I found an ambitious Maxis project about tribal civilization from the early 90’s that was never completed. It was like a murder mystery. Why had this project died? The hard drive was full of prototypes for a secret project, which turned out to be The Sims. Apparently, Maxis had been working on the game for a long time, and many aspects of it had been prototyped in isolation, including a 2.5d character animation system and editor, the motive and decision making AI, and a house editor. The code to the last prototype was clearly a hacked version of the SimCity 2000 engine. I found a program that used genetic algorithms to procedurally generate SimCity-style buildings with a blind watchmaker style interface. That program had clearly been written as efficiently as possible, not from a run-time point of view, but from an implementation standpoint. It was using the SimCity 2000 code base as a host organism for some rapid experimentation. Will’s imagination had clearly been running faster than proper software engineering practice allowed. All of this, plus the awesome array of prototypes the Spore concept team had been cranking out, made a big impression on me. I joined in, and contributed some of my own wacky prototypes to Spore.

What was going on here? What did all of this mean? Thinking back, I realized I had made two classic mistakes. First, my eyes were bigger than my stomach. The ambitious projects I had undertaken in the past “failed” because I made the mistake of not proving out the core ideas in prototypes. You can’t send a rocket to the moon if you haven’t first experimented with launching simple toy rockets. My sense is that the tribal civilization game died for similar reasons: its author had launched into an ambitious finished project without doing the proper research, sketches, and prototypes.

Secondly, my success/failure evaluation function had been wrong. While the code to the The Sims prototypes wasn’t in the final game, they had clearly informed the final product. All of my small “failures” were actually a series of small successes that had improved my design skills, and were in fact studies I could incorporate into larger projects. I had it backwards the whole time. My “successful,” but incomplete, large projects were the real failures. I had invested too much energy into large projects that would fail because I hadn’t done my homework. It’s a hard lesson to take, and one that most people probably have to learn the hard way.

So, I dusted off my ACM programming competition skills, which taught me to make tightly focused programs in minimal time, and with minimal frills. I became a better designer, really fast. I gained tons of design experience points by slaying so many gremlins.

I finished at Georgia Tech, and joined Spore. My vast collection of tiny student projects, bite-sized personal projects, and work prototypes I had made added up to a huge amount of experience and intuition. Compared to my peers, I had a tremendous amount of design experience, simply from writing, evaluating, and throwing away so many ideas. I witnessed good prototypes move mountains. I like to think of good prototypers as powerful ninjas who can drop into hard design challenges, or tedious design debates, and cut them to shreds with one swift movement of their prototyping blade.

Here are a few prototyping rules of thumb. Even with years of experience, I often find a prototype going nowhere, and can usually trace the problem to not following one of these rules:

• Always Ask a Question. Always ask a question, which will give you purpose, and have a hypothesis, which is a specific idea you are testing out. For example, you might be thinking about mouse based control schemes for a school of fish. Your question is: how do I control these fish with a mouse? A hypothesis might be: Flocking will make the fish move together, and every mouse click will drop an invisible “bomb” that will act as a repulser upon every fish’s steering AI, and take a few seconds to complete exploding. A good way to make sure you aren’t going to waste time implementing ideas you don’t actually have, which happens to me more often than I’d like, is to diagram the idea on paper first, and work out as many details with a pen as possible. This also speeds up writing the prototype.

• Stay Falsifiable. Just like good science, you must validate the results of your experiment. Did your hypothesis work? Does your fish flock control scheme feel good to you? Do your friends find that it feels good? Does it work in the context of your game idea? You can never user test and play test an idea too early. I have seen many cool ideas go down in flames because its owner was overprotective, didn’t think it was ready, didn’t believe the feedback they were getting, explained away people’s responses, or thought that only their opinion mattered. Eventually users will play with your work, and by then it will be much harder to fix the design. Incorporate the user into the design process as early as possible. Be honest with yourself and your players, and you will be richly rewarded. This one is easy for me, since as a designer, my main intent is to entertain and transform other people, so I’m always interested in what effect my work has on others. Watching people use what you make will also make you a smarter designer.

• Persuade and Inspire. We’re making entertainment and art — your prototype should be cool, fun, and excite people. If you and your peers are compelled, so will your players. On the flip-side, if something isn’t resonating with other people, perhaps your idea or approach should be reconsidered. Prototypes can be powerful persuasive devices. Keita Takahashi, the designer of Katamari Damacy, couldn’t convince anyone that rolling around a giant sticky ball would be fun. Until they played the prototype.

• Work Fast. Try to minimize time to your first “failure” (rejecting a hypothesis), and don’t be afraid to push the eject button. A classic error is to spend months working on an engine, architecture, or something else that has nothing to do with proving out your core design idea. Prototypes don’t need “engines.” Prototypes are slipshod machines held together by bubble gum and leftover bits of wire that test and prove simple ideas as quickly as possible. If you find yourself weeks or months into a project with only an engine, you’ve failed. Perhaps you need to articulate a specific gameplay idea to validate. For me, the ideal window of time to start and finish a prototype in (including design, implementation, testing, and iteration), is two days to two weeks. Anything longer than that sets off alarm bells.

• Work Economically. You’re making something small and beautiful, so invest development effort wisely. In order to work fast, you must stay small: don’t do too much at once, or you’ll never make progress. Be realistic. Here are some questions to ask yourself when you are considering how much effort to spend on proper engineering, art, interface design, or any aspect of your prototype. What’s the purpose of this prototype? Who will use it? What’s important? Look? Kinesthetics? Load time? Run time? Usability? Persuading your peers? Be a cheap, lazy, slothful programming bum. Just make it work, so you can test your idea. Don’t go above and beyond the call of duty in programming, art, or any other aspect of your prototype.

• Carefully decompose problems. Don’t bite off more than you have to at once. If you prototype all systems simultaneously you will fail — you can’t work fast, or reach any kind of conclusion. To build it all at once is to build the actual game, which is hard. The prototype designer’s job, like a good Go player, is to cut and separate the enemy stones  (your design problem) into small, weak groups that can be killed or manipulated at will. Wisely divide your problem into manageable pieces. You must be careful, because problems are sometimes connected in non-obvious ways, biting you later. Through practice, your designer’s intuition and experience will help you see the connected nature of the problem you are trying to subdivide, and make the most judicious cuts.