We're about to fire up the closed beta of Crashlands (I know, I know, FINALLY) which means we'll have a whole bunch of beta-testing novices providing feedback on the game. I needed to write up an article for them on how to give useful beta feedback, and figured it might be a useful reference for other game developers and game testers. SO HERE WE GO!
Understand what the feedback is for
Context is essential in providing useful feedback, because what is "useful" depends on the current state of the game (is it a prototype, or a finished product?), proximity to publication (months down the road, tomorrow, or is it already published?), your relationship to the developer (close friends, or never met?) and so on.
Remember: Feedback that isn't useful wastes everyone's time.
For example, if the developer hands you a game that is done and scheduled to release a month from now, they may want to know about game-breaking bugs (so they can make an emergency patch) but not want a laundry list of things that "could be a little better" (it's too late for that). But all this depends on the most important piece of context, which is what the developer is actually looking for.
Developers are people too, and often they don't think to explicitly tell you what kind of feedback they want. In that case, ASK! If they say "anything you think of!" then you should go do something else instead. Why? Because there is never a state of a game where any and all feedback is useful. A develop who hasn't carefully considered what value you can provide is going to be wasting your time.
Okay, so the most important first step to providing useful feedback is to focus on exactly what the developer is looking for (in retrospect, DUH, right?). But the hard part is in how to give feedback.
What useful feedback looks like
So you're playtesting a game and have stumbled across something you want to report. How can you do that in the most productive way? Here are the four components of good feedback:
Understand what the feedback is for
Context is essential in providing useful feedback, because what is "useful" depends on the current state of the game (is it a prototype, or a finished product?), proximity to publication (months down the road, tomorrow, or is it already published?), your relationship to the developer (close friends, or never met?) and so on.
Remember: Feedback that isn't useful wastes everyone's time.
For example, if the developer hands you a game that is done and scheduled to release a month from now, they may want to know about game-breaking bugs (so they can make an emergency patch) but not want a laundry list of things that "could be a little better" (it's too late for that). But all this depends on the most important piece of context, which is what the developer is actually looking for.
Developers are people too, and often they don't think to explicitly tell you what kind of feedback they want. In that case, ASK! If they say "anything you think of!" then you should go do something else instead. Why? Because there is never a state of a game where any and all feedback is useful. A develop who hasn't carefully considered what value you can provide is going to be wasting your time.
Okay, so the most important first step to providing useful feedback is to focus on exactly what the developer is looking for (in retrospect, DUH, right?). But the hard part is in how to give feedback.
What useful feedback looks like
So you're playtesting a game and have stumbled across something you want to report. How can you do that in the most productive way? Here are the four components of good feedback:
- What happened.
- Why you think the thing that happened is a problem.
- The exact steps required to make the thing happen, and how often it happens when you complete those steps.
- For 1-3, be as concise as possible.
What happened? Games are complicated machines: saying that something "didn't work" is both the most useless feedback there is and the most common we get. It's useless because there are a hundred ways for any particular thing to "not work." Those hundred ways fall into two broad categories: (1) you expected something to happen, but something else happened instead, (2) something happened, but you would have preferred that something else happened instead. In either case, the first part of good feedback is to explicitly and clearly state what happened, and what you either expected or wished had happened instead.
Why you think the thing that happened is a problem. Every gamer has a different play style and a different gaming experiences to draw on. This means that any two people might disagree whether or not something really is a problem. So try to explain as clearly as possible why the thing that happened is one. In some cases it's so obvious it doesn't need explaining (e.g. something that crashes the game). Otherwise you should assume that it isn't obvious.
How to make the thing happen. This is, by far, the most important component. If we can't reproduce the problem you describe, then we can't do anything about it. If we can't reproduce it quickly, then we'll waste time tracking down the problem. Tell us exactly what we can do do reproduce the problem. If possible, reproduce it yourself at least once before writing your report. Even better, do some experiments to see if you can do similar things that also cause the problem. For example, if you found a combat bug where striking an enemy causes the game to crash, see if striking a different enemy, or different enemy type, or using a different weapon, all cause the same crash. And then tell us what did and did not work.
Be concise. As a tester, you usually won't know what the developers need to know in order to reproduce and fix a problem you find. Therefore you should err on the tons-of-information side. But that comes at a steep cost for you and for the developers: long-winded reports take a long time to write and a long time to read! Your goal should be to use as few words as possible while precisely covering the above points. You don't need complete sentences, you don't need to pre-apologize if you're worried we'll take offense, and don't need to give us any backstory.
Examples of Feedback
Taking all of the above into account, let's take some made-up examples of Crashlands Beta feedback. For the beta, we consider the game to be content-complete and are only looking to polish the game for launch. This means that we're looking for bugs, balance, and usability issues.
Example Feedback #1
Bad version.I had been wandering the Savanna for what felt like my entire life, ferociously battling Hewgo and his minions whilst trying to placate the natives by solving their problems. As a native Woanopian, I must say that the developers' portrayal of the Tendraam is spot-on. Amazing work, you guys. Anywho, having wandered the Savanna -- as I was saying earlier -- I came across a fishing hole and cast my line deep into the watery depths. I celebrated vigorously as something grabbed the line, only to be dismayed by the fact of its skeleton-like appearance. It didn't belong there! I am loathe to admit that I found it so distasteful that I had to brush my teeth.
Good version. Savannah Fishing Holes produce skeletonized fatfish ~10% of the time [what happened and how to reproduce it]. These are healing items, but heal for WAY more than anything else, so I suspect they aren't supposed to be in the Savanna [why it's a problem].
Example Feedback #2
Bad version. The Spacetime Folder doesn't work.
Good version. 100% of the time when I click the Spacetime Folder in my hotbar nothing seems to happen, though the cooldown timer starts, implying that it should have done something. [what happened and how to reproduce it, obviously a problem]. I have tried it with and without the presence of a pet, in and out of combat, with various weapons and other hotbar items equipped, and with and without the presence of NPCs on screen. All other aspects of the game seem to behave normally. [more info about how to reproduce it, not worrying about which info is likely responsible]
Example Feedback #3
Bad version.I feel really terrible saying this, and I hope you don't take any offense. I certainly don't mean any! I mean, I love the story overall, but I just can't stand the Omelettes for Burlenon quest.
Good version. The Omelettes for Burlenon quest is too difficult [what happened]. I have enjoyed all other quests so far, but this one is frustratingly impossible. Anger Omelettes require killing Epic Shirks, which don't exist in the Savanna. [why it's a problem and, trivially, how to reproduce it]