01 Games User Research

Question 1

Common GUR methods

*common is relative to size of studio and culture

Answer 1

• Interviews
• Observation
• Surveys
• Telemetry / Analytics
• A/B Testing

Question 2

Interviews

Answer 2

A researcher asks individual players about the topic(s) of interest, while taking notes
and recording audio

Pros
* Generates contextual data that can stand alone, or explain other results
* Follow-up questions can help generate new insights
* An audio record provides a full account for later reference

Cons
* Researcher bias is not always obvious
* Can be difficult to prepare and run without experience
* Considered less rigorous or valid than quantitative data by some (they are wrong, but they might still think it)

Question 3

Obeservation

Answer 3

A researcher observes a tester’s play behavior while taking notes - Ideally, a second researcher facilitates the session, and the observer is elsewhere (e.g.,
watching on a separate screen)

Pros
* See how ‘typical’ players actually behave in the game
* Can be done early in development
* Helps understand what is happening

Cons
Presence of observers can bias results
Behavior requires interpretation -> why things are happening unclear
Player subjective experience mostly
unknown

Question 4

Surveys

Answer 4

A web form (or rarely, paper) with questions on players’ attitudes and experiences - May include open text questions, may include questionnaires, or both

Pros
* Relatively easy to deploy - Versatile, can be applied to many topics
* Standardized user experience questionnaires already exist

Cons
* Statistics knowledge is required to interpret quantitative results - Larger samples (N>100) are typically needed to test hypotheses

Question 5

Telementry (aka analytics)

Answer 5

Logging of behavioral data from players. This can be on any behavior that the system can ‘count’ (e.g., deaths, collisions, levels completed, time spent)

Pros
* Very granular and detailed
* Unobtrusive, does not interrupt gameplay
* Often very persuasive with stakeholders

Cons
* Non-trivial to implement data collection and data preparation
* Statistics knowledge is required to interpret quantitative results and visualize data
* Even with strong statistics knowledge, can be difficult to interpret meaning of results

Question 6

A/B testing

Answer 6

Players are randomly assigned to play one of two (or more) slightly different game variations
The behaviour of interest (e.g., spending, retention) is measured (via telemetry) to compare design builds or variations

Pros
* Generates a definitive answer
* Quantifies the influence of design changes
* Persuasive with stakeholders

Cons
* Requires a large enough player base for confidence in results
* Does not explain why results occurred – unless you have a strong hypothesis and clear study design

Question 7

Qualitative vs. Quantitative methods

Answer 7

Quantitative
* Collect numerical information, e.g., how much? how often?
* Aims at generalization across the population out of which the representative sample was drawn
* Typically requires large sample size (i.e., lots of testers)

Qualitative
* Collect descriptive information, e.g., how did this happen? why did you do this?
* The data obtained is thick and rich, and its analysis is about identifying data patterns
* Samples are typically small and purposive