04 Survey research 1/2

Question 1

What is a survey?

Answer 1

Survey: A cross-sectional design in relation to which data are collected predominantly by questionaire or by structured interview on more than one case and at a single point in time to collect a body of quantitative data

Question 2

What are the two forms of information in a survey?

Answer 2

Forms of information:
- Self reports: (respondent) –>individuals answer for themselves
- Key informant reports (“informant) –>one or a few answers for a larger propulation

Question 3

What is the typical survey process?

Answer 3

1.Selection of research variables
2.Selection of survey method
3.Questionaire design
4.Data collection
5.Measurement evaluation
6.Data analysis

Question 4

What is the psychology of survey response?

Answer 4

• Participants typically develop attitudes when answering a survey
• Survey questions trigger a cognitive process of response generation

1. Comprehension (Attent survey, deduce the survey´s content)
2. Retrieval (Memory search, long-term information enters short-term infomation)
3. Judgement: (Combination of info and formation judgement)
4. Response selection(Map judgement on offered response categories)
5. Response reporting (provide answer)

Question 5

What are the two measurement error of a survey?

Answer 5

random error:
- Expected value equal 0
- has no correlation with systematic error or true value
- threat to measureent reliability
–>(Hit the area, but point spread across whole area)

systematic error:
- expected value not equal 0
- often assumed no correlation with true value
- threat to measurement validity
(points are not on the area at all)

Question 6

When do systematic error generally happen?

Answer 6

systematic errors happen when there are a lot of factors that influence the measurement results,

–>i.e, such as social desirability that could lead to inflated answers

Question 7

What is the impact of the random error measurement on obersved correlationas between variables? (correlatiion coefficient)

Answer 7

Random error:
- low reliability, leads to a high variance random error
–>only denominator, decrease correlation coefficent

Random errors, have no impact on the sign of the correlation
- lead to underestimation of correlation
- No change of the sign of the correlation

Question 8

What is the impact of the systematic error measurement on obersved correlationas between variables? (correlatiion coefficient)

Answer 8

Systematic error:
- low validity, lead to high covariance of systematic error and high variance systematic error
–>both numerator and denominator:

Impact of systematic error is not predictable:
- Correlations between systematic error components can dishort the strength as well as the sign of the true correlation
–>high covariance: inflate (increase) correlation
–>low covariance: lowers correlation and eventuall flips the sign of the correlaton efficient

• IN the abscence of such correlations: systematic measurement error has the same consequences as random error

Question 9

WWhat are the sources of systematic errors in survey research?
(Common method bias, key informant bias, non-response bias)

Answer 9

Common method bias: Distortion of samples covariance structure that arises due to the fact that the same data source was used for measuring both IV and DV within a certain dependence structure

Key informant bias: Distortion of samples covariance structure that arises due to the fact that the data collection has taken place through key informants (provide info about larger social unit)

Non-response bias: Distortion of samples covariance structure that arises due to the fact that the structure of the effective sample, does not coincide with the structure of the original sample

Question 10

WWhat are the sources of systematic errors in survey research?
(Social desirability, response patterns)

Answer 10

Social desirability: Distortion of samples covariance structure that arises due to the fact that the tendency of respondents to reply in a manner thatt will be viewed favorably by others –>i.e. over-reporting goood behavior, undereporting bad behavior

Response patterns: Distortion of samples covariance structure that arises due to the fact that the respondents, regardless of the question content, favor certain response categories
a) Tendency to agree
b) tendency to Cross-middle points
c) tendency to give extreme responses

Question 11

What are the four relevant forms of construct validity?

Answer 11

• Predictive validity (demonstrate that question/scale has similar properties as expected
• Content validity/face validity: check if used items actually capture the right concept
• Convergent validity: items that capture same construct, have high relationships
• Discriminant validity: items that are suppose to measure different, have weak relationships

Question 12

What are the two reliability checks?

Answer 12

• reliability of the complete scale (construct reliability)
• reliability of single items (indicator reliability)

Question 13

What are the methods of first generation?

Answer 13

• Cronbach´s alpha and explaine variance in exploratory factor analysis at construc level
• Item-to-total correlation and factor loading in exploratory factor analysis at item level

Question 14

What are the methods of second generaton?

Answer 14

• Factor reliability and averrage variance explained at construct level
• Indicator reliability at indicator level (not relevant)

Question 15

What is cronbach´s alpha?

Answer 15

Issue not possible to measure error variance –>Cronbachs alpha

Idea: Variance shared by the items reflects the variance of the phenomenon

Cronbach´s alpha: how well survey items hang together and measure a commong underlying construct consistently

–>threshold: >70 –>that items good

Question 16

What is the interpretation of cronbach´s alpha?

Answer 16

• alpha correspnds to the correlation of the scale with an alternative scale with the same number of items
• alpha corresponds to the correlation of the scale with true score

Question 17

What is reliability?

Answer 17

Reliability: refers to the degree to which the observed variable measures the true value and i error free

–>Basic idea: the larger the variance of the true variable compared to the overall variance, the higher the reliability

Question 18

What is the issue with cronbach´s alpha?

Answer 18

issue: Cronbachs alpha does not provide any diagnostic information at the item level

–>Solution: Item-to-total correllation:
- assess the relationship between individual items and the total score of the scale.
- It provides insights into how well each item correlates with the overall score of the scale

Question 19

Briefly explain two options to increase measurement reliability in this situation? (Cronbachs alpha, item to toal correlation)

Answer 19

• Add further items to the scale (can potentially increase reliability depends on the item added
• Eliminate Item 5 , correlations between item 5 and the other items are dangeroulsly low

Question 20

Explain how a situation can arise in which a high Alpha is observed, even though the items do not validly measure the underlying construct?

Answer 20

When the scales measures something else: reliability does not automatically lead to validity, it is a necessary but not sufficient condition

Question 21

What is the process of questionnaire design?

Answer 21

• Decision about the survey content
• Decision about the question content
• Decision about the question format
• Decision about the question wording
• Decision about the question sequence
• Decision about the survey layout
• Pretest of the questionaire

Question 22

Question content: what are guidelines for scale development?

Answer 22

Guidelines for scale development:
– Ideally: four to six items
– Use the same expressions for the same issues
– Point out the necessity of repeating similar questions
– The entire width of the phenomenon should be covered by items (try to achieve representativeness

Question 23

What are the two formats possible in the decision about the question format?

Answer 23

open-ended vs. closed questions

Open-ended: respondents can respond in any way they want

Closed-questions: respondents choose an answer from a given list

Question 24

What are the advantages of open-ended questions?

Answer 24

• participants can answer in their own way
• allows for unusual answers
• little prior knowledge of researcher necessary

Question 25

What are the disadvantages of open-ended questions?

Answer 25

- quantification difficult - **larger effort** on the part of respondent needed, especially in the case of written surveys

Question 26

What are the advantages of closed questions?

Answer 26

- **straightforward** quantification - **simple to compare** - **support in interpretation** of questions - allow for **time-efficient question** “batteries”

Question 27

What are the disadvantages of closed questions?

Answer 27

- **loss of spontaneity** - **completeness** of answer categories often **hard to achieve**

Question 28

What are the 3 objectives in **question wording**?

Answer 28

Three objectives: - Simplicity - Unambiguousness - Neutrality

Question 29

**Question wording** how achieving simplicity?

Answer 29

**Rules for achieving simplicity:** – Short sentences (max. 15 words) – No pyramiding (e.g., through relative clauses) – Avoid negations – **Avoid foreign words**, if possible – Translate scientific or technical expressions in everyday language – Computations should be done by the researcher! – **Avoid unnecessary variations** in the format and structure of the questions

Question 30

**Question wording** how achieving unambiguousness?

Answer 30

**Instructions for achieving unambiguousness:** – **Avoid universal expressions** (e.g., “all”, “always”) – **Avoid vague quantifiers** (e.g., “often”, “many”) – Separate the definition and the question – Clearly define focal concepts – Clearly define the time span to which the question relates – Refer to only one issue – Use the same words for the same issue

Question 31

**Question wording** how achieving neutrality?

Answer 31

**Instructions for achieving neutrality**: - Avoid suggestive formulations - No anticipation of answers (e.g., “Do you also agree?”) - Avoid loaded words

Question 32

**Question sequence**, what iis importat about the cognitive process in survey response?

Answer 32

**Survey Response as Cognitive Process:** - Attitudes are developed instantly during survey responses. - Early questions activate information for later questions (potentially) - Involves assimilation and contrasting processes.

Question 33

What is the typical effect of **question sequence** in survey about **customer satisfaction?**

Answer 33

**Customer Satisfaction Surveys:** - Total satisfaction measured **first leads to emotional dominance** - Measured **last leads to cognitive dominance** (depends on prior answers). **Order preference depends on product involvement:** - **Low involvement:** Measure satisfaction first. - **High involvement:** Measure satisfaction last.

Question 34

What os the solution in question sequence to avoid biases?

Answer 34

**Solutions to Biases:** - Randomize question order. - Use "buffer" questions strategically. Carfully design the **sequence**: - Simple interesting openers, from general to specific - demographics at the end, sensitive questions later. - For **high relevance of "demand effects,**" assess dependent variable before the independent variable.

Question 35

Why is the question layout a critical sucess factor?

Answer 35

**Layout of questionnaire as critical success factor:** - “should look as sharp as your résumé” - Signals interest for the topic - Signals researcher’s trustworthiness (important in the case of confidential questions) - Facilitates completion of the questionnair

Question 36

What are elements of a careful layout?

Answer 36

**Elements of a careful layout:** – Accuracy – Clear and simple design – Clear identification of researcher and survey objective – High-quality paper and print (if printed)

Question 37

how to achieve the impression of short processing time?

Answer 37

**Create the impression of short processing time:** – Do not order questions too densely – Use readable letter size – Clear instructions – Clear visualization of the question flow

Question 38

What is a pretest?

Answer 38

**Pretest:** Testing of the questionnaire with selected / representative members of the sample

Question 39

What are the goals of a pretest?

Answer 39

**Goals:** - Identification of ambiguous questions and problems of understanding - Check for the completeness of answer categories - Identification of potential information gaps among respondents - Check the required time for completing the questionnaire - First indication of validity and reliability of the measuremen

Question 40

What are the methods of pretests?

Answer 40

**Methods:** – Debriefing: discussion after answering – Protocol method: discussion during answering – Think-aloud method: participants express all their thoughts while filling out the questionnaire **-->No best method**

Question 41

What are the two methods of survey data collection?

Answer 41

**Population of interest**: 1.**Census** (**complete** population 2.**Sample** (out of the population)

Question 42

What are the two sampling methods?

Answer 42

1.**Nonprobability sampling** 2.**Random** selection (probability sampling)

Question 43

What are the different methods of **random selection** (probability samplng)

Answer 43

- **Simple** random sampling: randomly select 10% out of all - **Stratified** sampling: divide in two groups (A) and (B) and select 10% from each - **Cluster** sampling: Create clusters consisting of A and B, that differ between - **Multi-stage** sampling: combination of different methods

Question 44

What are the methods of **Nonprobability sampling**?

Answer 44

- **Convenience** sampling: contact people most **convenient** (family, friends) - **Quota** sampling: goal reach quota=50%, quota needs to be achieved -->**difference to straitified**) - **Snowball** sampling: ask suitable people to **spread** the survey wiithin thier network - **Sampling** based on **typicality**: specifc targets are selectes, select individually who participates