Determinance screenshot

Indie advice: how to respond to beta feedback

Indie advice: how to respond to beta feedback

In the first two years of Frozen Endzone’s development only eight people had ever played it.  In the last six weeks, forty outsiders have been playing.  In two weeks we will launch the first public beta.  How we respond to the feedback from those testers will have a big impact on how well the game is received by the world at large.

 

Beta tester feedback is incredibly important.  Buried within it are the nuggets of information you need to make the small changes which will have the big impact.  You’ve done a great job getting your game this far – now finish the job.

 

But taking and responding to feedback is very difficult.  You love your game and you will get emotional about any criticism of it.  And there is usually a gulf between what testers say you need to do and what you actually need to do.  Each piece of feedback is an emotional and analytical journey – here’s my checklist for how to deal with each forum post or email.

 

1. Read it, then calm down, then read it again

You didn’t read it properly the first time.

 

2. Classify what kind of feedback it is

Here are some broad classes of feedback, with some notes on what to do with them:

 

“I don’t like the aesthetic”

You’re making a zombie game and this guy hates zombie games.

Move on – his feedback isn’t useful.

Relevance: zero
Frequency: high
What you’re feeling: “whaa no-one likes the aesthetic I’ve chosen”
What you should be feeling: There’s a pretty direct correlation between popularity of aesthetic and the number of other games using it.  So be happy – either you’re doing something popular or you’re doing something unusual.  Both of these things are positive.
Action to take: None.

 

“I love the aesthetic!  Why isn’t this a JRPG?”

This guy doesn’t like the genre of your game.  But he does like your aesthetic!  So take heart in that.

Relevance: zero
Frequency: high
What you’re feeling: “Hnnnnrgh… fuck…. off…”
What you should be feeling: Well, I have a hard time thinking anything but fuck off on this one.
Action to take: None.

 

“I can’t do this thing <which was taught perfectly in the tutorial>”

Oh this guy’s just an idiot who doesn’t pay attention to the tutorial.  Ignore.

WRONG.  This is an exceptionally useful piece of feedback you need to immediately jump on.  This guy has identified something which is hard to do.  He’s not going to be the only person who finds this hard.  This is your opportunity to make your game more accessible.  Maybe the thing could be made easier?  Maybe your control scheme for it is bad?  It’s crucial not to ignore this kind of feedback.

Relevance: high
Frequency: high
What you’re feeling: “I’m not aiming this game at people who can’t read instructions”
What you should be feeling: No one in the world reads instructions.
Action to take: Iterate this feature until people stop complaining that they can’t do it.

 

 

“Hey I did something really weird and the game broke”

Here is the golden rule of testing: if something happens once during testing then it will happen twelve thousand times on release day.  It doesn’t matter how weird the bug is – if the outcome of the bug is unacceptable, you must fix it now.

Relevance: high
Frequency: very high
What you’re feeling: “Oh god another bug to fix”
What you should be feeling: Thank god this came up now and not on launch day.
Action to take: Fix it.

 

 

“This feature you’re really proud of is shit – you need to get rid of it”

This is a great piece of feedback because, like most in-the-wild feedback, it contains something very important and something you should ignore.

Here is a direct translation of this feedback: “there is something wrong with this feature.”  It doesn’t mean anything more than that.  You must pay attention to this feedback, but the tester has made it hard by being too extreme.  People fall into the trap of not responding to this kind of feedback because they disagree with what the tester thinks you should do about it.  Ignore what the tester thinks you should do about it – just pay attention to the fact that you must do something about it.

Try to work out why he doesn’t like it – it could be badly taught.  It could be buggy.  It could be unclear.  Swallow your pride and iterate your feature more.

Relevance: high
Frequency: medium
What you’re feeling: “Pff.  You’re just a tester – you have no idea about game design.”
What you should be feeling: This is an opportunity to make my pet feature even better.
Action to take: Iterate.

 

“I was really expecting to like this game, and I really tried, but I just don’t”

This is a tough one.  Sometimes people just aren’t going to like your game – even if it’s great.  Frozen Synapse has been a huge critical success, but I still meet people in person who say “I respect it – but I don’t personally like playing it.”  You’re never going to please everyone in the world, and you can’t make an omelet without breaking some eggs.  Take it and move on.

Having said that, it’s always possible your game isn’t good enough yet.  If you’re hearing this a lot, maybe it’s something you need to think about properly.

Relevance: medium
Frequency: low (hopefully)
What you’re feeling: “I’m going to throw myself out of a window”
What you should be feeling: I’m going to get away from the game for a while and get some perspective.
Action to take: Generally, none.

 

“I’m enjoying this game – I’d really like to see you add feature x”

If a tester asks for a simple functional feature – something like the “opponent done turn” notification in FS – then I tend to implement if I can.

On larger things, I don’t tend to directly respond.  Maybe a feature a tester suggests will give me some inspiration, and getting a general idea of what people especially like about the game and what they’d like to see expanded is useful.  You have the best idea of your game’s overall direction – don’t get too distracted by new directions.

Many times testers come up with really great ideas which resonate with me straight away – that can be one of the best parts of indie game dev.

Relevance: medium
Frequency: medium
What you’re feeling: “Ooh! Maybe you should be able to talk to the monsters”
What you should be feeling: I don’t want to get derailed.
Action to take: take it in, and think on it.

 

“This game is fantastic!”

This is the feedback we all secretly hope we get deluged in.  A couple of things though:

First, I know you want your testers to fall in love with your game at first sight, but it doesn’t tend to happen.  With a released game a new player already has a context for your game from reviews and general buzz.  Without this people don’t tend to immediately fall head over heels – they need to get to know the game.  Don’t get depressed if you aren’t getting enough positive feedback in the first couple of days – look for what happens after a week or so.

Second, just as one guy not liking your game wasn’t cause to jump out of a window; one guy loving your game isn’t cause to release it straight away and expect 95% in every magazine.

Relevance: highish
Frequency: lowish
What you’re feeling: “Yes! I’VE WON INDIE GAME DEV”
What you should be feeling: well, that’s a pretty great feeling – enjoy it.
Action to take: Go get wasted.

 

<Crickets>

Finally, how do you respond to a lack of feedback?

A whole book could be written on this, but I’ll just go with a couple of things.

  • A good 50% of the people who signed up for your beta will never play it.  Just ignore this – it’s completely standard and is absolutely no reflection on your game.
  • If someone is still playing it after a week, then they like it.  It’s as simple as that.
  • If no-one plays it more than once, then that is a massive warning sign and there is something wrong with your game.  Don’t over-think this – the main way people tell you your game is bad is by not playing it.

 

Thanks for reading, and I hope this was useful to you.  If you have any comments please mail me at ian bat mode7games bot com.

 

Ian

 

Sorry, comments are closed.