Block 4 Flashcards
What is the difference between a usability goal and a user experience goal?
Usability tends to focus upon those aspects of an interactive product traditionally related to work, such as productivity and effectiveness. It also tends to focus explicitly on the user interface. The six usability goals discussed in Block 1 Unit 2 are: effectiveness, efficiency, safety, utility, learnability and memorability. User experience is a relatively new term, whose precise meaning is yet to be agreed. However, it relates to a wider set of concerns than those of usability, including issues such as whether an interactive product is fun to use, aesthetically pleasing, and so on. There are many user experience goals; the ten user experience goals concentrated on in Block 1 are: satisfying, enjoyable, fun, entertaining, helpful, motivating, aesthetically pleasing, supportive of creativity, rewarding and emotionally fulfilling.
Any kind of evaluation is guided either explicitly or implicitly by an evaluation approach. What is an evaluation approach? What are the three main evaluation approaches? When might each be used?
An evaluation approach is based on a distinct set of values and assumptions as to how evaluation should be conducted, either implicitly or explicitly. The three main evaluation approaches are: usability testing, field studies and analytical evaluation. Usability testing is an evaluation approach which uses a variety of methods to assess the usability and the experience of using an interactive product. Field studies are undertaken to learn about what users do naturally in their own settings, and how products mediate their activity. Analytical evaluation is used to predict problems that users may have with an interactive product. It is undertaken by experts, and users are generally not involved.
List the steps of the DECIDE framework, and for each step identify an activity associated with it.
1 Determine the overall goals that the evaluation addresses. An activity here is to decide the goals for the evaluation, and then restate the goals as statements which can be assessed in the evaluation.
2 Explore the specific questions to be answered. An activity here would be to turn the goals of evaluation into relevant questions for evaluation.
3 Choose the approach and methods to answer the questions. An activity here would be to choose an approach and methods that will be of help in finding answers to the evaluation questions.
4 Identify the practical issues that must be addressed. An activity here would be to select the participants for the evaluation.
5 Decide how to deal with the ethical issues. An activity here would be to adopt and follow a set of guidelines to ensure the evaluation is done ethically and protects participants’ rights.
6 Evaluate, analyse, interpret and present the data. An activity here would be to decide how best to present the findings of the evaluation to the development team.
Define each of the above usability goals.
Effectiveness refers to how good a product is at doing what it is supposed to do. Efficiency refers to the way in which a product supports users in carrying out their tasks. Safety involves protecting the user from dangerous conditions and undesirable situations. Utility refers to the extent to which the product provides the right kind of functionality, so that users can do what they need or want to do. Learnability refers to how easy a system is to learn to use. Memorability refers to how easy a system is to remember how to use, once learned.
For each of the usability goals, list the key questions you need to ask when establishing whether the particular goal has been satisfied.
Effectiveness. Is the product capable of allowing people to learn, carry out their work efficiently, access the information they need, or buy the goods they want?
Efficiency. Once users have learned how to use a product to carry out their tasks, can they sustain a high level of productivity?
Safety. What is the range of errors that are possible using the product and what measures are there to permit users to recover easily from them?
Utility. Does the product provide an appropriate set of functions that will enable users to carry out all their tasks in the way they want to do them?
Learnability. Is it possible for the user to work out how to use the product by exploring the interface and trying out certain actions? How hard will it be to learn the whole set of functions in this way?
Memorability. What kinds of interface support have been provided to help users remember how to carry out tasks, especially for products and operations they use infrequently?
Describe the key characteristics of an evaluation approach.
the role of users .
who controls the process and the relationship between evaluators and users during the evaluation .
the location of the evaluation .
when the evaluation is most useful .
the type of data collected and how it is analysed .
how the evaluation findings are fed back into the design process .
the philosophy and theory that underlies the evaluation approach.
Why would you want to observe users early in the interaction design
lifecycle
Why would you want to observe users later on in the interaction design
lifecycle?
Early in design, observation helps designers to understand users’ needs. Later in design, observation is used to assess whether the prototype of the product meets users’ needs. Observing users is useful at any time during product development.
Describe three types of observation.
Why and when might each be used for evaluating an interactive product?
Three types of observation are: direct observation in the field, direct observation in a controlled environment and indirect observation. Direct observation in the field would be most appropriate to evaluate a highfidelity prototype or an existing product that could be used in the target context. Direct observation in a controlled environment would be most appropriate to study specific aspects of a product. Indirect observation would be most appropriate to evaluate a high-fidelity prototype or an existing product.
What does it mean to use the think-aloud technique for gathering data?
The think-aloud technique involves the participant saying out loud everything they are thinking and trying to do while using a product.
What is a performance measure? What types of user performance measures are taken during a user test?
A performance measure is a quantitative measure related to the use of an interactive product. The following are example performance measures taken during a user test: the time taken to complete a typical task or tasks, the number of errors made while performing a task and the number of steps taken to complete a task.
Which data-gathering techniques are most commonly used in usability testing?
Direct observation, questionnaires, interviews and indirect observation through keystroke logging are the data-gathering techniques most often used in usability testing
Why are scripts useful for usability testing?
Scripts prepared in advance of a usability evaluation – and used for each participant – ensure that all participants are given the same information and are treated in the same way for the evaluation. This also ensures that it is possible to compare the results from participants.
Why is it important to select a representative sample of users whenever possible?
It is important to have a representative sample to ensure that the findings of the user test can be generalised to the rest of the user population. Selecting participants according to clear objectives helps evaluators to avoid unwanted bias. For example, if 90% of the participants testing a product for 9–12 year-olds were 12, it would not be representative of the full age range. The results of the test would be distorted by the large group of users at the top end of the age range
Suggest the kind of information that you would expect to include in a ‘welcome’ script for a user testing session
Your welcome script would include: the goal of the study and how the results will be used; an outline of what the participant will be expected to do (but not in detail as this will come later); the data recording that will be used, including video cameras, keystroke logging, etc.; an introduction to the environment including the presence of recording equipment and one-way mirrors; and a pointer to the informed consent form
Describe the differences between experiments and usability testing as discussed by Ben Shneiderman.
Experiments . Experiments test a hypothesis that has practical implications and theoretical results. . The results are of broad importance. . At least two conditions are tested. . Statistical tests are applied to the results to verify statistically significant differences. . The results confirm or refute the hypothesis being tested. . The experimental procedure is carefully described so that others can replicate it.
Usability test studies . Usability test studies involve only small numbers of users who carry out required tasks. .
Statistical analysis of the results is less important. . The goal is to refine a product as quickly as possible. . The outcome is a report to developers which identifies frequent problems and possibly suggests improvements.
