Week 9-Survey Methods

Question 1

Explain what Surveys are

Answer 1

■Systematic collection of info.
■Predetermined questions.
■Verbal or written.
■Questionnaires and structured
interviews.
■Survey = process.
■Questionnaire/interview = tool.

Question 2

What ways can you administer surveys?

Answer 2

-mail
-internet
-phone
-group
-structured interview
-personally

Question 3

Evaluate Mail surveys

Answer 3

+Convenient.
– Response rate/bias (Dillman, 2000).

Question 4

Evaluate internet surveys (email/website)

Answer 4

+Efficient and cheap.
+Convenient for participants.
+Large and diverse sample (including niche groups).
–Representativeness (less of an issue now).

Question 5

Evaluate phone surveys

Answer 5

+Some questions easier to ask.
+Large and diverse sample.
–Selection bias.
–Interviewer bias.

Question 6

Evaluate group surveys

Answer 6

+Captive audience.
+Large amount of data quickly.
–Privacy/anonymity.
–Pressure.

Question 7

Evaluate structured interviews

Answer 7

+Same questions, same order.
–Interviewer bias/Social context

Question 8

Evaluate Personal (FtF)

Answer 8

+Convenient/Large sample.
+Good response rate.
–Representativeness/Demand
Characteristics/ Questionnaire fatigue.

Question 9

What do Psychometric Tests measure?

Answer 9

■Ability/Aptitude: E.g. Numerical/verbal reasoning.
■Personal Qualities: E.g. Personality/attitudes.
–Employment suitability.

Question 10

What’s the history of survey methods?

Answer 10

■China from 2000 BC.
■Proficiency tests.
■605-1905 Chinese civil service exams, testing fundamental skills.
■Influenced development of exam/testing systems both in the East and in the West.

Question 11

Explain the history of Robert Yerkes (WW1) - Army Alpha and Beta tests (1917)

Answer 11

–Systematic method of evaluating intellectual and emotional functioning.
–Verbal/numerical ability, ability to follow directions etc.
–Capability of serving, job classification, leadership
potential.
–Beta test – non-verbal (for illiterate or foreign soldiers).

Question 12

What’s the flaw with some of the historical tests?

Answer 12

■Flawed
–Measure acculturation and
education.
–Longer in the US - better marks.
–Better education – better marks.
–Cultural and educational bias.

Question 13

What’s the Woodworth Personal Data Sheet?

Answer 13

–A test in World War 1 testing emotional stability - susceptibility to shell shock.
–First personality test which was highly influential in the development of later personality tests.

Question 14

What Modern-day tests are used?

Answer 14

■Stanford-Binet IQ test.
■Children with learning difficulties.
■ Used today for:
–Clinical and neurological assessment.
–Educational placement.
■First successful intelligence test.

Question 15

How is a questionnaire designed?

Answer 15

■Topic e.g., Beliefs/attitudes/Facts/ Behaviour.
■Type.
■Draft.
■Re-examine/revise (ideally with
experts).
■Pre-test/pilot study.
■Edit and detail procedures for use.

Question 16

What questions should be avoided in a questionnaire?

Answer 16

–Double-barrelled questions
■E.g. How satisfied are you with your essay marks and feedback?
–Loaded questions
■Bad: Do you think our hardworking and caring doctors and nurses should be paid more?
■Good: E.g. Do you think the NHS budget should be increased?
–Negative wording
■Bad: Do you think that students shouldn’t have to pay tuition
fees?
■Good: Do you think that students should pay tuition fees?

Question 17

Define open-ended/partially open-ended questions

Answer 17

■Open-ended:
–Qualitative, descriptive response.
■Partially Open-ended:
–Multiple choice plus Other
________

Question 18

Evaluate open-ended questionnaires

Answer 18

+Detailed answers.
+/-Quick to design, long time to
analyse.
–Subjective interpretation.
+Participant led (unanticipated data).

Question 19

Give examples of close-ended questionnaires

Answer 19

–True/False.
–Multiple Choice.
–Likert Scale.

Question 20

What are some of the problems with close-ended questionnaires?

Answer 20

–Guessing (T/F = 50/100).
–Unsubtle.
+/-Complex to design, but quick to
mark.
–Theory-led.

Question 21

Explain the Questionnaire Fallacy

Answer 21

The mistaken belief that a
questionnaire provides a true picture of what people
do and think.

Question 22

How do you create a Likert scale?

Answer 22

–Choose a specific topic.
–Formulate statements.
–Pilot test.

Question 23

True or false: high-frequency scales should be used for sub-specific groups

Answer 23

True

Question 24

What’s a Semantic Differential Scale?

Answer 24

Connotative meaning - Bi-polar
adjectives e.g., strong and weak and you rate the level you feel closest with.

Question 25

What are examples of Questionnaire Bias?

Answer 25

■Order Effects/Priming. ■Demand Characteristics. ■Social Desirability. ■Acquiescence. ■Extreme and Neutral Responses. ■Cultural.

Question 26

Define Order Effects

Answer 26

■Question order can affect responses. ■Detailed to General.

Question 27

What's Priming and how can it be overcome aswell as order effects?

Answer 27

Thinking about the answer to one question whilst answering the subsequent question. Overcome by: –Counter-balance questions. –2 versions: One order then another, compare results. –Randomise every question.

Question 28

Explain Social Desirability

Answer 28

■P's try to make themselves look as ‘good’ as possible e.g., Eysenck Personality Inventory ■Increase the positive or decrease the negative. ■Can be accentuated by the type of questionnaire/setting E.g., Drug use. ■Emphasise anonymity.

Question 29

What's Acquiescence?

Answer 29

■Yea-saying. ■Nay-saying. ■Balance of negative and positive questions.

Question 30

When are Extreme and Neutral Responses used?

Answer 30

■Mostly found in questionnaires using Likert Scales. –Neutral - not interested? Bored or fatigued? –Extreme – Yea/nay-saying? Topic? –Check raw data.

Question 31

Why does Cultural Bias arise?

Answer 31

■Early IQ tests culturally biased. ■Language that could be misunderstood. ■Various interpretations of words. ■Social attitudes/desirability differs e.g., Homosexuality. ■Complex or ambiguous academic terminology.

Question 32

Explain Psychometric Tests

Answer 32

■Avoids response bias. ■Implicit psychological processes. ■Unconscious aspects of human nature. ■Implicit vs explicit. ■Explicit = “Do you drink alcohol?” ■Implicit = “I prefer drinking with lots people” etc.

Question 33

What assumptions are there in Attitude Questionnaires?

Answer 33

–Attitudes can be expressed using verbal statements. –Statements will have the same meaning for all participants. –Attitudes expressed as verbal statements can be quantified.

Question 34

What are the problems with attitude questionnaires?

Answer 34

–Consistency with behaviour. –Social desirability. –Ambivalence. –Normative response bias. –Lie Scale (helps check honesty). E.g. ‘I never say things I later regret.’ OR ‘I always tell the truth.’

Question 35

What are Word Association Tests?

Answer 35

■First introduced by Francis Galton (who also founded psychometric psychology) ■Developed by Jung as a clinical diagnostic tool (but not just used in a clinical setting) ■Jung claimed that by using WAT he could identify: –Abnormal patterns of response. –Psychological complexes. –Intellectual and emotional deficiencies.

Question 36

What are the advantages and problems of WATs?

Answer 36

A: –Quick - easy to administer. –Predict prospective drug use. (Ames et al., 2007) P: –Colloquialism. –Standardised procedures (across cultures)? –Implicit?

Question 37

What are Implicit Tests?

Answer 37

■Implicit cognitive tasks. ■Infer attitude and beliefs from performance on different tasks. ■Often use reaction times. ■E.g. IAT (Implicit Association Test). –Automatic association between concepts. –Greenwald, McGhee, Sherry and Schwartz (1998). Age, gender, race etc.

Question 38

What are IATs?

Answer 38

■Computer- based. ■Categorise target concepts with an attribute/picture as quickly as possible. ■Faster pairing = stronger association.

Question 39

What are the criticisms of implicit tests like the IAT?

Answer 39

–Cultural views vs beliefs and attitudes. –Ecological validity - Attitude vs behaviour?

Question 40

What's a key element of Psychometric Tests and what types are there?

Answer 40

Reliability: –External – consistent across time and setting. –Internal – all items measure the same thing. –Test-retest reliability. –Split-half reliability.

Question 41

What's Classical Test Analysis?

Answer 41

■Observed score (X), true score (T), random error score (E). ■X = T + E. ■Random error (changes from person to person, time to time): –Failing to read questions properly. –Answering in a socially acceptable way. –Good/bad mood. –Tired or distracted.

Question 42

What's Systematic error + characteristics of test?

Answer 42

Predictable across people,settings, etc. ■Characteristic of test. –E.g. Happiness: How often do you... cinema, theatre, shopping, holiday, visit friends, exciting experiences? –Influence of participants’ wealth. –Problem with validity.

Question 43

Define validity + its types

Answer 43

The extent to which your measure is measuring what it is supposed to. –Content validity: Covers all behaviours/aspects. –Construct validity: Measures theoretical construct. –Criterion-oriented validity: Correlates with established measures.

Question 44

Explain Standardisation

Answer 44

–Standardised instructions and procedures. ■Established population norms: –Age – children, teens, adults etc. –IQ : UK average. ■Must relate to established tests and theories. ■Already validated, widely used tests. ■Social desirability.

Question 45

Questionnaires - Give Key Terms & Distinctions

Answer 45

■Knowledge Based: –Ability, aptitude, achievement. –Intelligence tests, and clinical assessment instruments. ■Person Based: –Personality, mood, attitude. –Differences, categorisation.

Question 46

Give 2 types of Response references

Answer 46

■Normative Reference Testing: –Scores are compared to the norm e.g., sample mean. ■Criterion Reference Testing: –Scores are compared to a pre- determined criteria.

Question 47

Explain Normative reference testing

Answer 47

–Compare to the norm. –Mean or Median splits. E.g., Number of units of alcohol per week. Above the mean/median = heavy drinkers. Below = light drinkers. Easy but arbitrary.

Question 48

Explain Criterion Reference Testing and how is it restrictive?

Answer 48

–Compare to pre-determined criteria. –At risk? Normal/pathological?  E.g. AUDIT - 11+ = hazardous drinking.  Beck Depression Inventory (BDI) 9+ = depressed. ■Restrictive: –Individual differences? –Non-clinical samples (student drinkers)?

Question 48

Explain Criterion Reference Testing and how is it restrictive?

Answer 48

–Compare to pre-determined criteria. –At risk? Normal/pathological?  E.g. AUDIT - 11+ = hazardous drinking.  Beck Depression Inventory (BDI) 9+ = depressed. ■Restrictive: –Individual differences? –Non-clinical samples (student drinkers)?