Lecture 9 - HCI Models and Evaluation Flashcards
1
Q
What are the goals of Evaluation? (3)
A
- Test hypothesis, usability and functionality of a system
- Assess effect of interface and interaction design on user
- Identify specific problems
2
Q
Where does evaluation take place? (3)
A
- Laboratory
- Field
- In collaboration with users
3
Q
When should evaluation be considered?
A
During all stages in the design life cycle
4
Q
Evaluation by system designers or experts is usually conducted when?
A
Early in the development cycle
5
Q
Evaluation by the end user is usually conducted when?
A
Later in the development life cycle
6
Q
What are the different evaluation types that take place during evaluation by system designers or experts? (4)
A
- Cognitive walkthrough
- Heuristic evaluation
- Model based evaluation
- Evaluation based on prior studies
7
Q
What does Cognitive walkthrough evaluate?
A
Evaluates a design on how well it supports a user in learning tasks
8
Q
What evaluation type is used for evaluating a product’s usability?
A
Cognitive walkthrough
9
Q
Who usually performs a cognitive walkthrough?
A
Expert in cognitive psychology
10
Q
How do experts evaluate a design using cognitive walkthrough?
A
They ‘walk through’ a design to identify potential problems using psychological principles
11
Q
What is used in cognitive walkthrough to guide analysis?
A
Forms
12
Q
For each task, cognitive walkthrough considers what? (3)
A
- What impact will interaction have on users?
- What cognitive processes are required?
- What learning problems may occur?
13
Q
TF: Cognitive walkthrough analysis focuses on whether the design leads the users to generate the correct goals
A
True
14
Q
How do you perform cognitive walkthrough on a small piece of an interface?
A
Alone, you (in your head) walkthrough to monitor the design as you work
15
Q
How do you perform cognitive walkthrough on a larger part of an interface?
A
As a group, you do a walkthrough for a complete task
16
Q
What 4 questions should be kept in mind when performing a cognitive walkthrough?
A
- Are users thinking what the designers thought they’d be thinking?
- Will users see the control (button, menu, switch, etc.) for the action?
- Once users find the control, will they recognize that it produces the effect they want?
- After the action is taken, will users understand the feedback they get, so they can go on to the next action with confidence?
17
Q
What are the three steps for performing a cognitive walkthrough?
A
- Identify the user goal you want to examine
- Identify the tasks you must complete to accomplish that goal
- Document the experience while completing the tasks
18
Q
What evaluation ‘debugs’ design?
A
Heuristic evaluation
19
Q
Who examines the design to see if it violates rules evaluated in Heuristic evaluation?
A
Experts
20
Q
How many people are involved in Heuristic evaluation?
A
3-5 inspectors
21
Q
In heuristic evaluation, how many usability problems does a single evaluator catch?
A
~ 35%
22
Q
In heuristic evaluation, how many usability problems do 5 evaluators catch?
A
75%
23
Q
What 2 things affect the number of usability problems found in a design?
A
- Difficulty of the interface being evaluated
- Expertise of the inspectors
24
Q
Why does Nielson recommend that individual evaluators inspect the interface alone for heuristic evaluation? (3)
A
- Evaluation not influenced by others
- Doing it alone results in greater variability in the kinds of errors found + unbiased
- No overhead required to organize group meetings
25
What are characteristics of self-guided heuristic evaluation? (3)
- Open-ended exploration
- Not necessarily task-directed
- Good for exploring diverse aspects of the interface, and follow potential pitfalls
26
What are characteristics of Scenario heuristic evaluation? (4)
- Step through interface using representative end user tasks
- Ensures problems identified in relevant portions of the interface
- Ensures that specific features of interest are evaluated
- Limits the scope of the evaluation - problems can be missed
27
What are the advantages of heuristic evaluation? (3)
- Few ethical & practical issues to consider since users not involved
- Quick: no need to find/schedule users
- Best experts have knowledge of application domain and users
28
What are disadvantages of heuristic evaluation? (4)
- Important problems may get missed (never find 100% of problems)
- Many trivial problems are often identified (minor issues such as button placement, unclear labels, etc.)
- Finding good experts and building consensus between them can be hard
- Not always valid since experts have biases and are not real users
29
What are the heuristics evaluated during heuristic evaluation? (10)
1. Visibility of system status
2. Match between the system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency use
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and documentation
30
What does the "Visibility of system status" heuristic define?
System should keep users informed about what is going on through appropriate feedback within reasonable time (i.e. 'you are here' indicator on mall map)
31
What does "Match between system and the real world" heuristic define?
System should speak the users' language. Words, phrases, concepts familiar to user rather than system-oriented terms (i.e. using 'save' instead of 'persist to disk')
32
What does the "User control and freedom" heuristic define?
Users often choose system functions by mistake and will need a clearly marked 'emergency exit' to leave the unwanted state without going through extended dialogue (i.e. cancel buttons)
33
What does the "Consistency and standards" heuristic define?
Users should not have to wonder whether different words, situations, or actions mean the same thing (i.e. using 'login' everywhere and not 'login' in one page, 'sign in' in another, 'access account' in another)
34
What does the "Error prevention" heuristic define?
Eliminate error-prone conditions or check for them and present users with a confirmation option before committing to an action (i.e. 'are you sure you want to delete' messages)
35
What are slips?
Unconscious errors caused by inattention
36
What are mistakes?
Conscious errors based on a mismatch between user's mental model and the design
37
What does the "Recognition rather than recall" heuristic define?
Minimize user's memory load by making objects, actions, and options visible (i.e. search bar with auto-complete suggestions)
38
What does the "Flexibility and efficiency of use" heuristic define?
Interface should work well for both beginners and experienced users. Experienced users should be able to use shortcuts to get things done quicker (i.e. Ctrl+C, Ctrl+V instead right clicking and selecting copy from menu)
39
What does the "Aesthetic and minimalist design" heuristic define?
Interfaces should not contain information which is irrelevant or rarely needed. Adding unnecessary information diminishes visibility of important information
40
What does the "Help users recognize, diagnose, and recover from errors" heuristic define?
Errors should not be expressed in plain language (no codes). Precisely indicate the problem & constructively suggest a solution (i.e. 'password should be 8 characters or more' instead of 'invalid password')
41
What does the "Help and documentation" heuristic define?
Even though system should be easy to use without documentation, it may be necessary to provide help and documentation (i.e. including a FAQs page)
42
What is Review-based evaluation?
Results from literature used to support or refute parts of design
43
What is Model-based evaluation?
Cognitive models used to filter design options
44
What are used to explain and predict user behaviour at the interface?
Cognitive models
45
What can be used to evaluate the usability of a product?
Cognitive models
46
What are Hierarchical models?
User task and goal structure: achieve goal by achieving subgoals (divide and conquer)
47
What are linguistic models?
User-system grammar (syntax/rules)
48
What are physical and device models?
Based on human motor skills
49
What is the Human Processor Model?
Cognitive modelling method used to calculate how long it takes to perform a certain task
50
What are the three main processors for the Human processor model?
- Cognitive (thinking, decision-making)
- Perceptual (senses, i.e. vision, hearing)
- Motor (typing, clicking)
51
TF: In Human processor model, each process has a cycle time
True
52
TF: In Human processor model, each memory does not have a decay time
False. It does
53
How can the Human processor model calculate the time it takes a user to perform a certain task?
By following the connections and using associated cycle or decay times
54
What is the Keystroke-level model (KLM)?
Model used to estimate how long it will take an expert user to perform a task using a computer interface
55
What are the 6 operators of KLM?
- K: Keystroke or button press
- P: Pointing to a target on a display with a mouse
- H: Homing (moving hand from mouse to keyboard, vice versa)
- D: Drawing straight-line segments
- M: Mentally preparing for executing physical actions)
56
What are the limitations of KLM? (5)
- Measures only time aspect of performance (execution time not time to acquire or learn a task)
- Only considers experts
- Only considers routine unit tasks
- Method has to be specified step-by-step
- Execution time has to be error-free
57
What are the 4 elements of GOMS?
- Goals
- Operators
- Methods
- Selection
58
What are Goals in GOMS?
What the user wants to achieve
59
What are Operators in GOMS?
The basic actions the user performs to reach the goal
60
What are Methods in GOMS?
Describe a procedure for accomplishing a goal
61
What are Selection Rules in GOMS?
The method the user uses to complete a goal (since there may be more than one)
62
What is an advantage of GOMS?
Analyst can easily estimate and calculate the time it takes to complete a particular interaction (only if the average times for each tasks were measured experimentally to a high degree of accuracy)
63
What are disadvantages of GOMS? (4)
- Only applies to skilled users
- Lack of account for errors
- Mental workload or fatigue are not addressed
- User personalities, habits, physical restrictions (disabilities) not accounted for
64
What is Touch-Level Model (TLM)?
KLM evolved for touchscreens and mobile devices
65
What are the 4 frequently used operations in mobile games?
- Tapping
- Pointing
- Dragging
- Flicking
66
What is Fitt's Law?
Law that states that the time to move to a target depends on:
- distance to the target (how far is it)
- size of the target (how big is it)
67
What is the formula for Fitt's Law?
Time = a + [b * log2(2D/W)]
68
TF: If a small object is made larger it becomes easier to click
True
69
TF: If a large object is made larger it becomes easier to click
False
70
Using Fitt's Law, where should frequently used elements, menus, buttons be placed to increase performance
Closer to a neutral cursor position
71
What is a Mental Model?
How you think something will work, based on your learning and experience
72
What model is based on belief, not facts?
Mental models
73
TF: It is not common to have a gap between designers' and users' mental models
False. It is
74
When does a good user experience happen?
When the conceptual model matches the mental model
75
How do you build a strong conceptual model? (5)
- Research and understand customer expectations
- Use terminologies they are familiar with
- Leverage the interface patterns they already use
- Test early, and iterate
- Re-evaluate the conceptual model at every step in the process
76
What are the two steps in Norman's Model of Interaction?
- Execution
- Evaluation
77
In Norman's Model of Interaction, what is Execution?
User forms a plan of action which is executed using the computer interface
78
In Norman's Model of Interaction, what is Evaluation?
After performing an action, the user checks the interface to see the result and decides what to do next based on that feedback
79
What are gulfs?
The gaps between the user and the interface
80
What is the gulf of execution?
The distance from the user to the physical system (i.e. You want to print a document but can't find the 'print' button)
81
What is the gulf of evaluation?
The distance from the physical system to the user (i.e. You click 'save' but no confirmation message is given)
82
What is a conceptual model?
The designers intended mental model for the user of the system (set of ideas of what it is and how it operates)
83
TF: A conceptual model describes interfaces in terms of keystrokes, mouse actions, or screen graphics
False. Defines abstractly in terms of of tasks
84
When creating a conceptual model, what are the two goals to keep in mind?
- Keep it as simple as possible
- Keep it as focused on the task-domain as possible
85
In Norman's model of interaction, what are the pros?
Useful in terms of understanding the interaction in a way that is clear and intuitive
86
In Norman's model of interaction, what are the cons? (2)
- Concerns user's view of the interaction
- Doesn't deal with the system's communication through the interface (how the system communicates back)
87
What is distributed cognition?
Theory stating that thinking and problem-solving doesn't just happen in the brain, but includes artifacts (tools, devices, documents, etc.), and other people (teammates, coworkers)
88
TF: Cognition is distributed between internal and external representations
True
89
TF: Cognition is not distributed across time
False. It is
90
TF: Cognition is distributed between individuals and groups
True
91
What are the three types of evaluation?
- Controlled settings that directly involve the users (usability and research labs)
- Natural settings involving the users (online communities, products used in public spaces)
- Any setting that doesn't directly involve users (consultants and researchers critique prototype)
92
What are Experimental Evaluations?
Controlled evaluation of a specific aspect of interactive behaviour
93
What factors are involved in Experimental Evaluation? (4)
- Subjects - Who?
- Variables - Things to modify and measure
- Hypothesis - What do you want to show
- Experimental Design - How are you going to do it?
94
In Experimental Evaluation, you are aiming to investigate hypotheses about how designs affect... (3)
- User performance (time, error rate)
- Satisfaction
- Emotions
95
In Experimental Evaluation, what is the independent variable?
Characteristic changed to produce different conditions (i.e. interface style, number of menu items)
96
In Experimental Evaluation, what is the dependent variable?
Characteristic measured in the experiment (i.e. time taken, number of errors)
97
In Experimental Evaluation, how is the hypothesis framed in terms of?
In terms of independent and dependent variables
(i.e. 'error rate will increase as font size decreases')
98
What is the null hypothesis?
Hypothesis stating there is no difference between conditions (i.e. states that for a given hypothesis, the change in condition did not affect outcome)
99
TF: We aim to prove the null hypothesis
False. Aim to disprove it
100
What is Protocol Analysis?
Method used to understand how people think while performing a task
101
What is a Post-Task Walkthrough?
When a transcript is played back to participant for comments
102
What are the two ways of performing a post-task walkthrough?
- Immediately - fresh in user mind
- Delayed - gives time for evaluator to find questions
103
What are advantages of post-task walkthroughs? (2)
- Analyst has time to focus on relevant incidents
- Avoid excessive interruption of task
104
What are disadvantages of post-task walkthroughs? (2)
- Lack of freshness
- May be post-hoc interpretation of events (guess or rationalize why they did something)
105
What are Living Labs?
Real-world environment where people use and interact with technology as part of their normal daily life
106
What are physiological measurements?
Physical measurements taken from user's body to understand their reaction to an interface (i.e. heart rate, electrical activity in brain, etc.)
107
What reflects the amount of cognitive processing a display requires?
Eye movement
108
What are the three measurements included in eye tracking?
- Fixation: eye maintains a stable position
- Saccades: rapid eye movement from one point of interest to another
- Scan paths: moving from one target straight to another with short fixation is ideal
109
What is fixation linked to?
Attention
110
What is a Gaze plot?
Shows the 'visual path' of an individual
111
What does a bubble represent in a Gaze plot?
A fixation
112
What does the bubble size denote in a Gaze plot?
Length or intensity of the fixation
