Week 3 (Advanced Issues in Survey Research Methods)

Question 1

Types of Survey Design

Answer 1

-Questionnaire
-Observations
-Interviews

Question 2

What does survey measure/look at

Answer 2

-Associations/correlations
-But NOT causal relationship, as doesn’t involve manipulation
-Aims to describe individuals and offer predictions about them based on socio-demographic information

Question 3

Stages of survey design

Answer 3

-Theory
-Hypothesis
-Operationalisation of concepts
-Survey studies/ Experimental design
-Selection of participants
-Data collection
-Data analysis
-Findings

Question 4

Open Questions (Survey)

Answer 4

-Participants are free to answer how they want
-Potentially more representative of person’s true opinions
-More true to real life (higher ecological validity)
-Often highlights aspects of research topic the researcher may not have thought of
-Difficult to analyse

Question 5

Closed Questions (Survey)

Answer 5

-Participants have to select a pre-specified response (1-7)
-Useful for statistical analysis/ diagrams
-Much easier to score and analyse
-More susceptible to designer bias

Question 6

Checklist for standardised measures

Answer 6

-Item generation: How were the items generated? Who was consulted?
-What evidence is there that the measure is reliable?
-Is the measure valid?

Question 7

Types of reliability

Answer 7

-Test-retest reliability
-Inter-rater reliability
-Inter-method reliability
-Internal consistency

Question 8

Types of validity

Answer 8

-Predictive validity
-Concurrent validity
-Convergent validity
-Discriminant validity

Question 9

Questions to ask when planning scale (Kryiazos & Stalkikas, 2018)

Answer 9

-How many items are needed?
-What response scale is most appropriate?
-How will the scale be scored?
-Which psychometric model is most appropriate
-What item evaluation process is suitable
-How to administer the test

Question 10

Likert scale

Answer 10

-Used to express extent of agreement, frequency etc.
-Offer different strengths of opinion
-Often used in standardised measures to produce scores
-Direct form of questioning

Question 11

Semantic Differential

Answer 11

-First developed by Osgood in 1952
-More indirect
-Takes advantage of peoples ability to think metaphorically
-Asks people to rate something on a number of pairs dimensions

Question 12

Categorical Answers

Answer 12

-Short questions offer a range of options or categories
-Often used for demographic information
-Not typically used on standardised measures

Question 13

Common errors when writing questions

Answer 13

-Ambiguous questions
-Technical terminology
-Leading questions
-Hypothetical questions
-Patronising tone
-Value judgements
-Context effects
-Multiple content/ double-barrelled questions
-Hidden assumptions

Question 14

Ambiguous questions

Answer 14

The format or focus of the required answer is unclear

Question 15

Technical terminology

Answer 15

Words may be unfamiliar

Question 16

Leading questions

Answer 16

Leads participants to a particular response

Question 17

Hypothetical questions

Answer 17

Response dependent on hypothetical condition

Question 18

Value judgements

Answer 18

Personal bias affecting wording

Question 19

Context effects

Answer 19

Issues relating to other questions on the form

Question 20

Multiple content/ double-barrelled

Answer 20

Asks about more than one thing

Question 21

Item Response Theory

Answer 21

-Provides an estimation of the discrimination parameter; how well an item functions as a measure of a latent construct (similar to factor loading)
-Allows assessment of difficulty/ severity threshold in the latent construct continuum

Question 22

Measurement invariance

Answer 22

-Captures the degree to which your measure is testing the same thing across conditions
-Increasingly popular
-Useful for testing differences across time or across conditions

Question 23

Test-retest reliability

Answer 23

If I measure your height today, will it be the same as yesterday

Question 24

Inter-rater reliability

Answer 24

If I measure you, will your height be the same as when someone else measured you

Question 25

Inter-method (or parallel forms) reliability

Answer 25

Will your height be the same if I use a tape measure as when I used a ruler

Question 26

Internal consistency

Answer 26

Did you respond to similar questions in a similar way

Question 27

Split-half technique (Internal consistency)

Answer 27

You test consistency across two halves -E.g, if questionnaire has 20 items, you test if first 10 responses correlate with second 10 questions.

Question 28

Construct validity

Answer 28

Is this a valid measure of tapping into the construct you are attempting to measure

Question 29

Content Validity

Answer 29

The extent to which the domain of interest is adequately represented by the scale items

Question 30

Predictive Validity

Answer 30

Is able to predict future behaviour

Question 31

Concurrent validity

Answer 31

Is score related to another criterion of this construct tested at the same time

Question 32

Convergent validity

Answer 32

Is the score related to other measures of the same construct

Question 33

Discriminant validity

Answer 33

Is the score different from measures that theoretically measure something else

Question 34

Floor-ceiling effect

Answer 34

The extent to which the scores cluster near the low/high extreme on the scale

Question 35

Mail Survey Pros and Cons

Answer 35

Pros -Low costs -No interviewer bias -Suitable for sensitive topics Cons -Low completion rates -Prone to non-response bias -Errors arising from participants misunderstanding questions

Question 36

Personal interviews Pros and Cons

Answer 36

Pros -More control and flexibility over how the survey is administered -Can make use of computer technology Cons -Interviewer bias = influences responses or records them incorrectly -High costs (time/financial)

Question 37

Telephone surveys Pros and Cons

Answer 37

Pros -Cost effective & time-effective -Can make use of computer technology Cons -Sampling bias -Interviewer bias -Low response rate

Question 38

Internet survey Pros and Cons

Answer 38

Pros -Low costs -Access to large, diverse, geographically remote, or underrepresented populations -Quick data collection -Can be used in conjunction with specialist software to automate recruitment, data entry, and incipient data analysis Cons -Sampling bias -Non-response bias -Lower response rates compared to mail, personal and telephone surveys -Lack of control over the research environment