CS2003 - Week 25 - Surveys and Questionnaires

Question 1

What is the difference between a survey and a questionnaire?

Answer 1

A survey is a structured method to gather information, using various methods (e.g., interviews, phone, online); A questionnaire is a specific tool within a survey: a set of structured questions (printed or digital) for collecting responses

Question 2

What are the main types of questions in a questionnaire?

Answer 2

Demographic (e.g., age, gender); Factual (behavior or ownership); Non-factual (attitudes, beliefs, intentions); Also includes: Open-ended questions (free text); Closed questions (multiple choice, yes/no)

Question 3

When are questionnaires most useful in usability engineering?

Answer 3

Best for gathering data about current systems; Useful for feedback on potential new features; Less suitable for early design stages when needs are unknown

Question 4

What are the three core metrics of usability?

Answer 4

Effectiveness – Can users complete tasks?; Efficiency – How quickly and easily?; Satisfaction – Are users happy with the system?

Question 5

What do validity and reliability mean in surveys?

Answer 5

Validity: Are we measuring the right thing?; Reliability: Are results consistent over time?; Combinations: ✅ High Validity + High Reliability = Ideal; ⚠️ High Validity + Low Reliability = Inconsistent; ⚠️ Low Validity + High Reliability = Misleading; ❌ Low Validity + Low Reliability = Poor data

Question 6

What are key considerations when designing a usability questionnaire?

Answer 6

Use pre-validated tools when possible; Keep demographic questions short, place at the end; Ensure ethical standards (e.g., anonymity, data minimalism)

Question 7

What are common issues with factual questions and how can they be addressed?

Answer 7

Recall bias and social desirability bias; Mitigate using clear time frames and specific response sets

Question 8

What are the concerns when using non-factual (attitude) questions?

Answer 8

Risk of bias in wording or interpretation; Avoid ambiguity, complex phrases, or leading language

Question 9

How should attitude scales be constructed for best results?

Answer 9

Use 5–10 point scales; Mix positive and negative items; Types: Likert scales, semantic differentials

Question 10

What are the main steps in developing a new questionnaire scale?

Answer 10

Conduct qualitative research; Create initial item pool; Pilot test; Use factor analysis for refinement; ⚠️ Note: This is time-consuming, not ideal for short projects