Evaluation Flashcards
Why should evaluation be conducted? When should evaluation be conducted?
Evaluation should be conducted in order to find out any issues with the prototype so that further refinement can be done.
It should be done in the design process and be as early and as often as possible.
What are the differences between empirical and predictive evaluation? What are their pros and cons?
Empirical: Verifiable by means of observation or experiment with real users
Pros: More “realistic” data, lab studies replicable
Cons: Expensive for lab/time-consuming for lab
Predictive: Done by experts in domain to find potential problems and give prescriptive feedback. Works best when they haven’t used earlier prototype and understand user perspectives. Either heuristic evaluation or cognitive walkthrough.
What is a “think aloud” usability study? How do you conduct one? Is it empirical or predictive? Do you conduct it in the lab or the field?
For the “think-aloud” method, users have to complete certain tasks for an objective, and for each action that they do, they have to speak out their thought process while completing the tasks. The evaluator takes notes of problems faced, record video/audio and analyses the data afterwards. They should also facilitate user responsiveness by asking questions like “tell me what you are thinking”. They should not help users do the tasks or give solutions.
It is empirical, it can be conducted in both the field and in the lab, depending on the type of data we wish to gather.
What is the DECIDE framework? How is it useful?
Determine aims and goals
Evaluate questions
Choose evaluation approach and methods
Identify practical issues
Decide on ethical issues
Evaluate, interpret and present findings
It is useful as a guideline to follow when trying to perform an evaluation study.
What is a heuristic evaluation? How do you conduct one?
Nielsen’s heuristic evaluation is based off the 10 heuristics criteria and it goes through step by step. There are at least 2 passes for each evaluator. First pass would be get feel for flow and scope of system, and second pass would be focused on specific elements. Each evaluator would produce list of problems and explain why with reference to heuristics.
What is a cognitive walkthrough? How do you conduct one?
A cognitive walkthrough assesses learnability and usability through simulation of the way users explore and become familiar with the system.
Steps to conducting:
1. Construct carefully designed tasks from screen mock-ups
2. Walk through activities required to go from one screen to another
3. Review actions needed for task, attempt to predict how users would behave and what problems they will encounter
Requirements:
1. Description of users and their backgrounds
2. Description of task user is to perform
3. Complete list of the actions required to complete task
4. Prototype or description of system
Methodology:
1. Step through action sequence (Action 1, Response A, B, Action 2, Response A)
2. For each sequence, ask four questions and try to construct believability story.
The 4 Questions:
1. Will user be trying to produce effect?
2. Will user notice the action is available?
3. Will user know it’s the right action for the desired effect?
4. Will user understand the feedback?
