Chapter 43 Flashcards
PoHCI, week 7 (17 cards)
What is an experiment?
A study in which an intervention is introduced to observe its effects. An experimenter changes
something, or intervenes, while keeping everything else the same, and observes the effect of the change
What is independent variables?
Something that is systematically varied in an intervention. Example: Changing the color of a button or the age group of the user
What is dependent variables?
The effects of the intervention, variables that depend on the intervention.
For example: Taskcompletion time or errors
What does nuisance factors describe?
If users were first asked to carry out a task A with a user interface, and then task B. The order of the interventions interacts with users’ learning. Any measurements in task B would be affected by what
users learned in task A.
In general, its factors that influence the situation under study and potentially affect the dependent variables
What are hypotheses?
Statements that connect variation in independent variables with expectations about variation in the
dependent variables
What types of research questions are there?
- Empirical questions: phenomena and effects in human-computer interaction
- Constructive questions: the ability to construct systems and designs with desirable properties
- Conceptual questions: relationships between theoretical constructs that represent interaction
The purpose of an experiment can either be for setting objectives for design (constructive), or to
distinguish between competing theories (conceptual)
What are the pros and cons of experiments?
Maximize precision, but at the expense of generalizability and realism
Precise manipulation of tasks and settings
Detailed data collection
Allow us to control external factors
Time-efficient
Investigate technology without actually deploying it
How should you choose your participants in an experiment?
Chosen participants should be representative of the user group. 12-20 is the least amount of participants recommended
What is within- and between-participants?
Within-participants: The independent variable is varied for each participant, so each participant
reviews each independent variable
Between-participants: The independent variable is the same for each participant group, but they only
review one independent variable
What is field evaluations?
To bring out prototypes of interactive systems in their thought use setting; the field.
Why should we do field evaluations
The prototype is tested in a real context
We can capture collaborative, communicative, and material practices
We can understand how systems work with users’ real tasks and motivations
We can observe social and organizational impacts
What is pilot studies?
To partially implement an interactive system and put it in use to learn about its use. Then, those learnings are used to improve the design of the system and fully implement it
What is deployment studies?
To gather information about a fully functional system and improve it as part of its maintenance
What is descriptive statistics?
Describe relationships between variables in the dataset
Summary statistics: Mean, median, variance
Visualization: Histograms, scatter plots, line plots
What is inferential statistics?
Draw conclusions about the population from the sample
Confidence intervals
Hypothesis testing
How do you explain the results of an experiment?
Quantitative results alone is insufficient
Other sources could be qualitative data (verbal protocols, video recordings, interviews) and theories
(cognitive theories, communication theories)
What is the pros and cons of field vs. lab studies?
Field studies:
- Higher realism
- Better socio-technical fit assessment
Can be of long duration
- Lower precision
- Higher cost
- Requires more robust systems
Lab studies:
- Higher precision
- Lower cost
- Can evaluate unusual tasks
- Can use rough prototypes
- Lower realism
- May miss socio-technical issues
