Brainscape's Knowledge GenomeTM

Browse over 1 million classes created by top students, professors, publishers, and experts.


See full index

Graphics MST > Lecture 13 (Evaluation Techniques) > Flashcards

Lecture 13 (Evaluation Techniques) Flashcards

1
Q

Why involve humans?

A

Computer applications are used by humans so it is important to evaluate them

  • Computer graphics are inherently human-centered
  • Presence of images, animations, and interaction that will influence behavior
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is Evaluation?

A
  • Tests usability and functionality of a system
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

Where does Evaluation occur?

A
  • In the lab, field, and/or in collaboration with users
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

When should Evaluation take place?

A
  • Evaluation should be considered at all stages of the design lifecycle
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What are the goals of evaluation?

A
  • Test hypotheses on users
  • Assess extent of system functionality
  • Assess effect of different features on the user
  • Identify specific problems and attempt to solve them
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the two Evaluation Styles?

A
  • Laboratory studies

- Field studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What are the pros and cons of Laboratory studies?

A

Pros:

  • Specialist equipment available (e.g. eye-tracking software)
  • Uninterrupted/controlled environment

Cons:

  • Lack of context of real-life use (not as realistic)
  • Difficult to observe several users cooperating
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

When is it appropriate to conduct Lab studies for evaluation?

A

If system location is dangerous or impractical for constrained single user system to allow controlled manipulation

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

What are the pros and cons of Field studies?

A

Pros:

  • Natural environment
  • Context retained (though observation may alter it)
  • Longitudinal studies possible

Disadvantages:

  • Distractions
  • Noise
  • Generally less controlled
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

When is it appropriate to conduct Field studies for evaluation?

A

Where context is crucial for longitudinal studies

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What are the three Design Evaluation Techniques?

A
  • Cognitive walkthrough
  • Heuristic evaluation
  • Review-based Evaluation
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

What is Cognitive Walkthrough?

A

A design evaluation technique

  • Evaluates design on how well it supports users in learning the task
  • Usually performed by experts in cognitive psychology
  • The expert “walks through” the design to identify potential problems using psychological principles
  • Forms used to guide analysis
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What does a Cognitive Walkthrough consider?

A
  • What impact will interaction have on the user?
  • What cognitive processes are required?
  • What learning problems may occur?
  • Does the design lead the user to generate the correct goals?
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What is Heuristic Evaluation?

A

A design evaluation technique

  • Use this when actual user testing isn’t able to be conducted
  • Used to debug the design
  • Usability criteria (heuristics) are identified
  • Design examined by experts to see if these are violated
  • Needs high level of expertise
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What is Review-based Evaluation?

A

A design evaluation technique

  • Uses literature to support or refute parts of the design
  • Use this when actual user testing isn’t able to be conducted
How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are the three Observational Methods?

A
  • Think Aloud
  • Cooperative Evaluation
  • Post-task Walkthroughs
17
Q

What is Think Aloud?

A

Observational method

  • User observed performing a task
  • User is asked to describe what they are doing and why, what they are thinking of etc.
18
Q

What are the pros and cons of Think Aloud method?

A

Pros:
- Simplicity

Cons:

  • Subjective (users might have varying levels of experience)
  • Selective
  • Acts of describing may alter task performance (user might not be good at multitasking)
19
Q

What is Cooperative Evaluation?

A

Observational method

  • Variation on Think Aloud (user collaborates with evaluator)
  • Both user and evaluator can ask each other questions throughout
    • e.g. Why did you make this mistake? What went wrong?
20
Q

What are the pros and cons of Cooperative Evaluation?

A

Pros:

  • Less constrained and easier to use
  • User is encouraged to criticise system
  • Clarification is possible

Cons:
- More interruption

21
Q

What is Post-task Walkthroughs?

A

Observational method

  • User reacts on action after the event
  • Used to “fill in” intention
  • Necessary in cases where Think Aloud is not possible
22
Q

What are the pros and cons of Post-task Walkthroughs?

A

Pros:

  • Analyst has time to focus on relevant incidents
  • Avoid excessive interruptions of task

Cons:

  • Lack of freshness (users may forget things)
  • May be post-hoc interpretations of events (users might fake answers if they forget things)
23
Q

Methods of Recording Data in evaluation?

A
  • Paper and Pencil (cheap)
  • Audio (good for think aloud)
  • Video / Screen capture (accurate and realistic, needs special equipment, obtrusive)
  • Computer logging (automatic and unobtrustive, requires large amounts of data to analyze, also difficult to analyze)
  • User notebooks (subjective, useful insights, good for longitudinal studies)
  • Mixed use and practice
24
Q

What are some Physiological Methods for evaluation?

A
  • Eye Tracking

- Physiological Measurements

25
Q

What is Physiological Measurements in evaluation? How may they be helpful?

A
  • Emotional response linked to physical changes
  • These may help determine a user’s reaction to an interface
    e. g Heart activity, activity in brain, etc.
26
Q

What are some Query Techniques for evaluation?

A
  • Interviews

- Questionnaires

27
Q

What are Interviews? And what are the pros and cons?

A

Query Technique
- Analyst questions users one-on-one with prepared questions

Pros:

  • Cheap
  • Issues can be further explored
  • Can identify unanticipated problems

Cons:

  • Very subjective
  • Time consuming
28
Q

What are Questionnaires? And what are the pros and cons?

A

Query Technique
- Set of fixed questions given to users

Pros:

  • Quick and reaches a large user group
  • Can be analyzed more rigorously
  • Can measure something specific
  • Measurements can be compared

Cons:

  • Less flexible
  • Can’t go back to user and ask why they answered something
  • Can’t identify unanticipated problems
  • Needs careful design of the questionnaire
29
Q

How to plan out a Questionnaire?

A
  • Decide what you want to measure
  • Find standardized questionnaire or create your own
  • Deploy and collect data
  • Analyse data
30
Q

How to choose an evaluation method?

A

When in process
- Design vs Implementation

Style of evaluation
- Laboratory vs field

How objective
- Subjective vs Objective

Types of measures
- Qualitative vs quantitative

Level of information
- High level vs Low level

Level of interference
- Obtrusive vs unobtrusive

Resources available
- Time, users, equipment, expertise

Graphics MST (14 decks)

Brainscape helps you reach your goals faster, through stronger study habits.
© 2024 Bold Learning Solutions. Terms and Conditions