Heuristic Evaluation (Quiz 5) Flashcards Preview

Comp Sci 570 > Heuristic Evaluation (Quiz 5) > Flashcards

Flashcards in Heuristic Evaluation (Quiz 5) Deck (40):

Heuristic Evalution

A rational or logical (as opposed to empirical) expert review method where heuristics are used to find and highlight usability problems.

Involves a small team of evaluators to evaluate an interface based on recognized usability principles.



"rule of thumb"


HE In a Nutshell

1. Individual Evaluation
2. Group Process
3. Write a report


Individual Evaluation

Each evaluator compares design with known usability principles of rules of thumb (heuristics)

Writes down all the problems in Usability Aspect Report (UAR)


Group Process

1. Group combines the problems

2. Assigns severity ratings


HE method

Inspection method


Usability Testing (UT) method

Empirical method


HE people involved

Best with 3-5 evaluators, one manager


UT people involved

Best with 5 users, one experimenter


Should HE evaluators be left without help?

Evaluators should NOT be left without help


Should UT users be left without help?

Users are left without help to struggle


Duration of HE sessions

1-4 hrs/session


Duration of UT sessions

0.5-1 hr/session


How does HE find problems?

Finds hardest-to-find problems, but may miss common problems


How does UT find problems?

UT finds most common problems, including conceptual model problems


Neilsen's 10 Heuristics

1. Visibility of system status
2. Match between system and the real world
3. User control and freedom
4. Consistency and standards
5. Error prevention
6. Recognition rather than recall
7. Flexibility and efficiency of user
8. Aesthetic and minimalist design
9. Help users recognize, diagnose, and recover from errors
10. Help and communication


1. Visibility

“The system should always keep users informed about what is going on, through appropriate feedback within reasonable time.”

Refers to both visibility of system status and use of feedback.

Any time the user wonders what state the system is in, or the result of some action, this is a visibility violation.


2. Real World Match

“The system should speak the users’ language, with words, phrases and concepts familiar to the user, rather than system-oriented terms. Follow real-world conventions, making information appear in a natural and logical order.”

Refers to word and language choice, conceptual model, metaphor, mapping, and sequencing problems.


3. User In Control

“Users often choose system functions by mistake and will need a clearly marked ‘emergency exit’ to leave the unwanted state without having to go through an extended dialogue. Support undo and redo.”

Not just for navigation exits, but for getting out of any situation or state.


4. Consistency

“Users should not have to wonder whether different words, situations, or actions mean the same thing. Follow platform conventions.”


Internal consistency

consistency throughout the same product.


External consistency

consistency with other products in its class


5. Error Prevention

“Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.”

Try to commit errors and see how they are handled. Could they have been prevented?


6. Recognition not Recall

“Minimize the user’s memory load by making objects, actions, and options visible. The user should not have to remember information from one part of the dialogue to another. Instructions for use of the system should be visible or easily retrievable whenever appropriate.”

Deals with visibility of features and information (where to find things), whereas #1 deals with visibility of system status and feedback (what is going on).

Problems with affordances may go here


Should the user ever carry a memory load?



Example of problem with affordance

hidden affordance means you have to remember where to act; false affordance means you have to remember it is a fake!


7. Flexibility and Efficiency

“Accelerators—unseen by the novice user—may often speed up the interaction for the expert user such that the system can cater to both inexperienced and experienced users. Allow users to tailor frequent actions.”


Examples of Flexibility and Efficiency

Concerns anywhere users have repetitive actions that must be done manually.

Also concerns allowing multiple ways to do things.


8. Aesthetic Design

“Dialogues should not contain information which is irrelevant or rarely needed. Every extra unit of information in a dialogue competes with the relevant units of information and diminishes their relative visibility.”


Issues with Aesthetic Design

1. clutter

2. overload of visual field

3. visual noise

4. distracting animations



9. Error Recovery

“Error messages should be expressed in plain language (no codes), precisely indicate the problem, and constructively suggest a solution.”

Error prevention (#5) is about preventing errors before they occur. This is about after they occur.


10. Help and Documentation

Even though it is better if the system can be used without documentation, it may be necessary to provide help and documentation. Any such information should be easy to search, focused on the user’s task, list concrete steps to be carried out, and not be too large.

This does not mean that the user must be able to ask for help on every single item.

Remember, this is different from a “user manual.”


Individual Evaluation

Briefing (HE method, Domain, Scenario)

Evaluation by each individual.

Passes through the interface:

1. Read all of the heuristics and their short descriptions beforehand. Queue them up in your mind.

2. Have a cheat sheet of the 10 heuristics in front of you.

3. Inspect flow. Inspect each screen, one at a time against heuristics.

4. Make a general pass, getting a feel for the interaction. Note obvious violations.

5. Make a second detailed pass, focusing on things that caught your attention earlier.

HE is not a single-user empirical test


More than One Evaluator

1. A single person will not be able to find all usability problems

2. Different people find different usability problems

3. Successful evaluators may find both easy and hard problems


Evaluator Findings

1. Multiple evaluators independently produce a list of usability problems.

2. Aggregate findings into a single list of problems and the heuristics violated. Eliminate redundancies and clarify.

3. Independently review the list and assign a severity rating to each problem.

4. Create a summary report is that includes the average severity rating for each problem. The evaluators and design team then go through a debriefing session, discuss the problems, potential fixes, and add fix ratings to the summary report.

5. UI Redesign / Next Prototype


Severity and Fixability Ratings

Used to allocate resources to fix problems and estimates of need for more usability effort.

Combination of frequency, impact, and persistence



Combination of:

1. Frequency - Will the problem be common or rare?

2. Impact - Easy or difficult for users to overcome?

3. Persistence - One time that users overcome once they know about it, or repeatedly bothered, even if they know about it?


How do we deal with persistence if problem leads users to abandon the application?

Do not rate low persistence if problem leads users to abandon the application.


Advantages of HE

"Discount usability engineering"

1. Don't need to identify tasks, activities

2. Can identify low-cost, high-value fixes

3. Can expose problems user testing doesn't expose (e.g. with expert functions)

4. Provides a language for justifying usability recommendations

5. Can be used with any level of prototyping


Disadvantages of HE

1. Non-empirical, thus un-validated

2. Unreliable (different people identify different problems)

3. Needs usability experts

4. Problems unconnected with tasks

5. Heuristics may be hard to apply to new technology

6. Coordination costs