week 3: advanced issues in survey design Flashcards
(27 cards)
what are the 8 stages of survey design (Hayes, 2000)?
- theory
- hypothesis
- operationalisation of concepts
- survey study
- selection of participants
- data collection
- data analysis
- findings
what is reliability
to what extent is the measure consistent or dependable
what is validity
to what extent is it measuring what it is supposed to measure
what are the 4 main types of reliability?
- test-retest
- inter-rater
- inter-method
- internal consistency
what are the 4 main types of validity?
- predictive
- concurrent
- convergent
- discriminant
what are some common errors when writing questions for survey research?
- ambiguous questions
- technical terminology
- leading questions
- hypothetical questions
- patronising tone
- value judgements
- context effects
- multiple content/double-barrelled questions
- hidden assumptions
what is exploratory factor analysis?
- statistical model specifying a structure underlying the data
- methodology: computational procedures that enable us to analyse data and reveal underlying structures in an unconstrained way
- statistics-led approach
What is confirmatory factor analysis?
- identify whether the items in a questionnaire capture the constructs we want to measure
- theory-led approach
What is exploratory structural equation modeling (ESEM)?
combines features of exploratory factor analysis and confirmatory factor analysis
What is item response theory?
provides an estimation of the discrimination parameter i.e., how well an item functions as a measure of a latent construct
- allows the assessment of difficulty thresholds in the latent construct continuum
what is measurement invariance (equivalence)?
captures the degree to which your measure is testing the same thing across conditions
what is construct validity?
is the tool a valid means of assessing the construct you are attempting to measure
what is content validity?
the extent to which the domain of interest is adequately represented by the scale items
what is predictive validity?
is the tool able to predict behaviour in the future?
what is concurrent validity?
is the score related to another criterion of this construct, tested at the same time
what is convergent validity?
is the score related to other measures of the same construct?
what is discriminant validity?
is the score different from measures that theoretically measure something else
what are floor-ceiling effects?
the extent to which scores cluster near the low (floor) and high (ceiling) extreme on the scale
when building questionnaire, what factors should be considered?
- respondent motivation
- length
- density
- presentation
- question order
advantages of mail surveys
- low cost
- no interviewer bias
- suitable for sensitive topics
disadvantages of mail surveys
- low completion rates (~30%)
- prone to non-response bias
- errors arising from participants misunderstanding questions
advantages of face-to-face interviews
- more control and flexibility over how the survey is administered
- can make use of computer technology - CAPI (computer assisted personal interview)
advantages of telephone surveys
- cost and time effective
- can make use of computer technology - CATI (computer assisted telephone interview)
