Planning

Question 1

Steps in evaluating a prototype

Answer 1

build
state hypothesis
identify variables
design experiment
choose popularion
apply for review if human subjects
run pilot studies
run experiment
perform analysis
draw conclusions

Question 2

elements of HCI evaluation plan

Answer 2

objectives - questions to answer
participants - plan for their selection
experiment design - what will they do, how connects to research questions
limitations - scope of methods etc.

Question 3

answerable

Answer 3

All research questions should have expected answers (aka hypotheses)

Question 4

non-triviality

Answer 4

Is the answer to the research question well-known or easily discovered?
If so, it is not a good research question for postgraduate study.

Question 5

testability

Answer 5

Can we find the answer to a research question?

Specifically, can we construct an experiment or conduct an analysis to determine the answer?

Question 6

Falsifiability

Answer 6

Falsifiability means that it MUST be possible to prove a deductive research question to be false. Some condition must exist where, if it is true, implies a negative answer to the hypothesis.

Question 7

Reproducibility

Answer 7

The answer to a research question should not be vague or applicable to specific data or a specific time (not specified in the question itself).

Question 8

Variable types in experiment design - name and explain 5

Answer 8

independent - you control them and vary to see effect
dependent
hypothesis
control variables - fixed throughout
random variables - attributes that do vary but that you dont deliberately vary - includes external variables

Question 9

common dependent variables in HCI

Answer 9

performance metrics
- task success , completion time, errors, efficiency
satisfaction metrics

Question 10

two types of data

Answer 10

qualitative

quantitative

Question 11

external validity

Answer 11

participants are representative of target audience

study conditions reflect real world use

Question 12

internal validity

Answer 12

experiment set up should not favor any one condition

Question 13

how to do limitations and reflections

Answer 13

reflect onmethods

Question 14

sources of limites

Answer 14

population representation
methods
environment
prototype fidelity
time 
money
bias

Question 15

WOZ studies

Answer 15

user
interface
obsercer
computer
interface elements

Question 16

Likert scale

Answer 16

repondents rate level of agreement with a given statement

Question 17

Considerations when using Likert scale

Answer 17

feel like quant measures
nuances in language make unsuitable for multilingual
scalse are not evenly distibuted
quick and dirty vs methodologically sound

Question 18

what are focus groups

Answer 18

Interviewing people in a group – usually 3-7 people
Participants can respond to and inspire one another
Lots of opinions at once!

Question 19

what are workshops

Answer 19

Larger groups (10-20) – with planned interactions
Locus for training, creativity
90min to multi-day workshops
Great for sharing of ideas and data
Community-based data collection

Question 20

key goals for moderating workshops and focus groups

Answer 20

A safe space for participants to interact
All participants can contribute
Interaction around ideas

Question 21

qualities of a good moderator for workshops and focus group

Answer 21

Has a plan
Listens well!
Observes and responds to cues from participants
Invites participation from all
Understands overall goal
Maintains topic focus - respectfully

Question 22

between subject design

Answer 22

different groups of users use different interfaces

Question 23

within subject design

Answer 23

everyone uses all interfaces (fred and whoever and betty and wilma use both interfaces)

Question 24

pros and cons of between subject design

Answer 24

Participants cannot compare conditions (+/-)
Can collect more data for a given condition (+)
Need more participants (-)

Often the choice is forced
If the task is time-consuming each subject will only be able to complete one condition  Between Subjects is necessary

Question 25

pros and conds of within subject design

Answer 25

Compare one person across conditions to isolate effects of individuals (+)
Requires fewer participants, less overhead (+)
Fatigue effects (-)
Bias due to ordering/learning effects (-)

If the task is short, filling a one hour session will consume several conditions  Within Subjects is natural