How we use player feedback

We get a lot of requests and suggestions from our players for changes to our games. Sometimes we do make those changes (especially during development and testing) and so this is an invaluable form of feedback. However, more often than not, we will not make those changes, and I figured it was worth a moment to explain why.

What criterion do we use to decide which changes to make and which to leave by the curb?

"All you'd have to do..."
This phrase is a running joke in our studio, and it comes up often. The situation is this: one of the three of us has a cool idea for something that will have to be implemented by a different one of the three of us. The cool idea is conceptually simple in the mind of its inventor, and so the presentation begins with "all we'd have to do is...". But it turns out that it only sounds simple because the inventor doesn't know what the idea requires practically.

Translating even a simple idea into code can be a hard problem, and ideas in game design are rarely simple. Games are crazy-complicated, non-linear things that change in time and have tons of inter-dependencies between parts. Making a change in one place may require making many other changes to seemingly unrelated functions.

Sometimes making a change is the only good move. In Crashlands, for example, during a design discussion I asked "Why not just get rid of the inventory system altogether?" We all agreed it was the right idea, but that little "just" turned into a couple weeks of development time, and the aftereffects of that change still show up in other Crashlands game systems every once in while (as bugs). Why? Because the inventory system is tethered to just about every other game system in some way, and things can fall through the cracks.

We only make changes when the awesomeness of the change clearly outweighs the amount of work involved to make it, and this is a rare thing (we throw out 90% of our own ideas!).

Coding for the average player

While we have great testers and do a ton of testing ourselves, once a game is released into the hands of a million players even the rarest of bugs will happen to SOMEONE. We know this, and so as developers we write our code while constantly thinking of all of the weird things that a player could ask the code to do. And we try to protect the game from the rare events (the "edge cases") so that even unlikely player behavior won't crash the game.

The problem is that edge cases are hard to predict, and they are usually complicated to code around, so that handling edge cases is WAY more work (and therefore way more development time) than handling the "average" case.

The same idea holds true for game features. As game developers and designers, our goal is to create an amazing gameplay experience for our players. However, every player has his or her own ideas about what makes a game "amazing," and rarely will any two players agree. We deal with this by making games that the three of us love, and just figure that if we love them so will at least some other people.

It turns out that most suggestions are things we've already considered and discarded, and we simply discard them again. Others are things that indeed would make the game better, but very rarely will they make the game much better, and even more rarely yet will they make the game much better for most players.

If a suggested change would indeed be a big improvement for many players, and looks like a doable amount of work, then we bring it against the final check.

Is this what we should be working on?

Games can ALWAYS be better. When we publish a game, we do so with full awareness of a jillion things that could have made that game a little better. There will always be rare bugs to work out, content to add, parameters to tweak. We could take any one of our games and iterate on it forever. And it is seductive for us as a studio to fall into that trap, because improving something that our players enjoy is immediately emotionally gratifying, and it's way easier than making a new game.

But here's the kicker: every moment spent iterating on one game is a moment taken away from developing the next one. For example, in the time we've spent patching Flop Rocket and our other old games, we could have made at least one entirely new game roughly the same size as Quadropus Rampage! That's one game that now will never exist!

And so we are constantly checking ourselves with the question: "Sure, thing X would make that old game better, but is it worth stopping/delaying development on the next game to change it?" Combine that with the simple point that updates to old games provide almost zero revenue when compared to launches of new games, and we have a strong incentive to always and only be developing the next game.

Our goal as a studio is to make games - not a singular ever-improving game (at least until we make an MMO. HEH!). By passing incoming suggestions through this gauntlet of questions:

1. How much work does this take?
2. Does this make the game much better for most people?
3. How does this effect our other titles in development?