Lecture 9 Surveys and Questionaire

Question 1

What is the difference between a survey and a questionnaire?

Answer 1

A survey is a structured method to gather information, using various methods (e.g., interviews, phone, online).

A questionnaire is a specific tool within a survey: a set of structured questions (printed or digital) for collecting responses

Question 2

What are the main types of questions in a questionnaire?

Answer 2

Demographic (e.g., age, gender)

Factual (behavior or ownership)

Non-factual (attitudes, beliefs, intentions)
Also includes:

Open-ended questions (free text)

Closed questions (multiple choice, yes/no)

Question 3

When are questionnaires most useful in usability engineering?

Answer 3

Best for gathering data about current systems

Useful for feedback on potential new features

Less suitable for early design stages when needs are unknown

Question 4

What are the three core metrics of usability?

Answer 4

Effectiveness – Can users complete tasks?

Efficiency – How quickly and easily?

Satisfaction – Are users happy with the system?

Question 5

What do validity and reliability mean in surveys?

Answer 5

Validity: Are we measuring the right thing?

Reliability: Are results consistent over time?

Combinations:

✅ High Validity + High Reliability = Ideal

⚠️ High Validity + Low Reliability = Inconsistent

⚠️ Low Validity + High Reliability = Misleading

❌ Low Validity + Low Reliability = Poor data

Question 6

What are key considerations when designing a usability questionnaire?

Answer 6

Use pre-validated tools when possible

Keep demographic questions short, place at the end

Ensure ethical standards (e.g., anonymity, data minimalism)

Question 7

What are common issues with factual questions and how can they be addressed?

Answer 7

Recall bias and social desirability bias

Mitigate using clear time frames and specific response sets

Question 8

What are the concerns when using non-factual (attitude) questions?

Answer 8

Risk of bias in wording or interpretation

Avoid ambiguity, complex phrases, or leading language

Question 9

How should attitude scales be constructed for best results?

Answer 9

Use 5–10 point scales

Mix positive and negative items

Types: Likert scales, semantic differentials

Question 10

What are the main steps in developing a new questionnaire scale?

Answer 10

Conduct qualitative research

Create initial item pool

Pilot test

Use factor analysis for refinement
⚠️ Note: This is time-consuming, not ideal for short projects