Test 2 Flashcards
(26 cards)
Define systematic observational research.
viewing and recording of a predetermined set of behaviors. There is something that the researcher is specifically looking for – makes it more scientific. Ex: a researcher observed people in everyday context and he was observing language and he had people wear recording devices. The researcher then transcribed and wrote down everything the participant said. They were interested in if there were gender differences between men and women and the amount of words spoken. They counted words (predetermined behavior). There was no statistically significant difference.
Compare and contrast types of observational research (laboratory, participant, naturalistic).
Laboratory research: observational research in a lab setting.
Participant observation: observational research in which the observer participates with those being observed. So researcher is not a bystander.
Naturalistic observation: observational research in which the observer observes events as they occur in a natural setting without interfering in any way. Not in a lab.
Describe benefits and problems of different types of observational research - laboratory
Problems:
- May lack mundane realism: is the situation similar to a real-world situation? Ex. When you’re going to get in a fight with your partner, do you drive to a local university, walk up to the third floor and then start fighting? No… In lab research, it’s not very similar to situations or settings people would normally find themselves in. So many times we’re giving up mundane realism when we’re in a lab.
- Strive for ecological validity (or psychological realism): is the psychological situation experienced similar to the psychological situation experienced in a real-world situation? As long as the psychological experience is the same psychological experience that you would have in a real world situation, it doesn’t matter that it’s not a real world situation because your psychological experience is the same. So we tend to not be too concerned with mundane realism as long as we have psychological realism.
- Reactivity: might behave differently in a lab because they know they are being observed and decide to be on their best behavior.
Benefits:
- Control: you have a controlled environment. You don’t have to deal with other things that are going on.
- May be able to collect data you wouldn’t otherwise have been able to:
- May be better than self-report: instead of just asking questions about themselves, you can observe them and that might be stronger data.
Describe benefits and problems of different types of observational research - participant observation
Problems:
- Familiarity may reduce objectivity: when you become familiar with people you might not be as objective as if you were a passive observer.
- May be difficult to find situations in which to use this method
- Reactivity:
Benefits:
- Better than self-report: participants may not have been honest if you asked them about their sexual behavior on a self-report measure.
Describe benefits and problems of different types of observational research - naturalistic observation
Problems:
- Ethics: what about watching people in like the bathroom? Not ethical. What usually distinguish naturalistic research that is ethical and that isn’t ethical would be are people expecting to be observed? So ex. If you go to a museum/in a classroom/hockey game, people generally say that that’s ethically okay because people expect to be observed in those settings. Where it crosses into unethical is in situations where people expect privacy.
When researchers are reporting this information, no participants names/identifying information are being reported so the participants do have some privacy.
Even though you’re not collecting informed consent, and not generally doing debriefing, it’s still going to go through an IRB – so the IRB is going to be a way to make it ethical.
Benefits:
- May be better than self-report:
- Aren’t typically concerned about reactivity
Explain the research process for observational research.
- Operationalize your definitions: how are you defining your variables?
- Develop your coding system: how are you measuring your variables?
- duration recording: recording the elapsed time during which a behavior occurs.
- frequency-count recording: recording each time a target behavior occurs during a time frame.
- observation schedule: paper-and-pencil or electronic form where the observers note the particulars of the behavior or phenomenon they observe. Usually give behaviors numbers (ex. smile = 1, frown = 2 etc.) because it’s easier to take notes on.
- Pick your sample: what sorts of criteria are you going to have for your sample? Nonprobability samples/probability samples – what type? Who are you interested in?
- Train observers
- blind observations: observers are trained to code behaviors but are uniformed about the purpose of the study, they’re not going to know the hypothesis.
- intra-observer reliability: extent to which an observer consistently codes a variable. Are the observers own ratings/coding consistent?
- inter-observer reliability: extent to which observers coding the same phenomenon agree. You want the observers to be consistent with each other. 2+ people observe and record what they see, assessed with r or Kappa, scores closer to 1 = better. Normally we want over 80 % agreement between observers.
- Collect data
- interval recording: breaks down an observation period into equal, smaller time periods and records behaviors during those time periods. Ex. Look for a behavior for one minute every 10 minutes.
- continuous recording: records all behaviors the whole time. Ex. Look at a behavior during a whole dinner.
- Analyze data
- categorical variable (nominal data – it doesn’t have order or meaning, ex. Hair color, gender, year in college): a way to classify data into distinct categories. If you have a number, you’re using it just to label it.
- frequency distribution: a summary of how often the individual values (or ranges or values) for a variable occur. Ex. How many sophomores there are at UM. - continuous variable: variables with a number of different values between two given points. The numbers have order, ex. Height, weight. Can do more statistically analysis with this data than with categorical – most often you do mean, median and mode.
- mode: most frequent occurring score.
- mean: average.
- median: middle score.
- if you had multiple observers, you’ll calculate and report a measure of agreement between the observers. Common measure is Cohen’s kappa coefficient. You want high numbers, above 80 %. When you do the analysis, you would normally just use one set of data so you’ll have t come up with a way to determine which data to use. Maybe use the person’s data who has the most experience.
- Present data
Differentiate between open-ended and closed-ended questions.
Open-ended: they can give you any answer they want. Can give you more information. Might have to do coding to analyze it, so it takes longer to analyze.
Ex: “how do you feel when you are the center of attention?”
Closed-ended: answers by agreeing or disagreeing. It’s easy to analyze data after you’ve have closed-ended data. Participants are constrained and can’t answer whatever they want.
Ex: “I really like to be the center of attention” and they answer like this: 1 – strongly disagree, 2,3,4,5 – strongly agree.
Describe scales.
Scale: measurement strategy for assigning a number to represent the degree to which a person possesses or exhibits the target variable. Includes multiple questions, closed-ended questions.
Likert scale: a scale where a participant evaluates a series of statements using a set of predetermined response options; the responses are summed to represent the overall measurement for the variable. Numbers are related to the answers (1-5, or 1-7). You add numbers up to get an overall score.
Understand some problems with developing good scale items.
- Leading questions: if a question contains anything that could be an answer. “do you think I did a bad job?”, “do you think I did a good job?” vs. “how did I do?” (this is not going to push people to a specific answer.
- Double-barreled: concerning because you’re asking two questions at the same time. Ex: “is research methods your favorite class and do you have two legs?”
- Negative wording: when you put negative words in your question. Ex: “If you weren’t to advocate not doing your quizzes”. You should only use one negative word at the time if you need to use negative wording. Appropriate to ask “I do not like to go to parties” because there’s only one negative word in there.
- Question order: people like to respond in a consistent manner, ex: if I asked you first a question about if you’re an optimist, and then asked you how likely it would be that you’re a victim of a crime, the first question could influence the way you answer the second question. If you said yes, I’m an optimist, you might feel like you need to answer that you think you’re unlikely to be a victim of a crime.
One option to fix this is to change question wording. Maybe put question about crime before asking of they’re optimists. You can also change the order of the answers, so sometimes start with strongly agree and sometimes with strongly disagree.
Describe problems with participant responses.
- Response sets (yea-saying, fence sitting): yea-saying means they respond yes all the way down, you don’t get meaningful data. No-saying means the opposite. Fence sitting means answering in the middle (neutral). One way to catch this is if we do reverse coding. If someone says strongly agree to “I’m a sociable person” and then later say strongly agree to “I do not like to go parties” = they should disagree to the later, so you can catch this. Also don’t offer a fence to sit on, don’t offer a middle response. But sometimes people might legitimately feel like they are in the middle so you might force people to respond in a way that doesn’t fit them.
- Social desirability: answer in a way that makes you look good. To get around this: make things anonymous.
- Faulty introspection: when people think that they are aware of the way they think and feel but they are actually not aware about what they think and feel. Ex: when asked about who was to blame for your latest argument, there’s a good chance you would say that the other person is more to blame, and if other person is asked he will say you’re to blame. So our feelings and attitudes might not be correct.
- Memory errors: you can’t remember everything accurately.
Be able to calculate scores for Likert scales.
Reverse-coding: flip the scoring. A scoring strategy where more negative response alternative are assigned higher values and more positive response alternatives are assigned lower numerical values. You can also flip the questions, ex: “I do not like wild parties” -> “I like wild parties”.
If you’re concerned about people not really reading the questions, you can put in “attention-checks” where you can say “for this questions, answer ‘strongly agree’. So you tell them how to answer, and you can see if they answered the questions the way you told them to, and see if their data is useful.
Explain internal consistency reliability and test-retest reliability.
Internal consistency reliability: reliability within the survey/questionnaire itself. If there are similar items (questions/statements), that people should respond similarly to similar items regardless of how the question is phrased.
- Consistency of answers (in measure that contains several items) regardless of how question is phrased.
- Assessed with Cronbach’s a.
- a = .70 or higher is considered adequate.
- high scores are good scores.
- Ex: self-control scale: “I am good at resisting temptation.”, “I am able to work effectively toward long-term goals.” “I refuse things that are bad for me”. Etc. 1 = strongly disagree, 5 = strongly agree. If there’s good reliability, people should be responding similarly to these questions.
Test-retest reliability: administer same test twice with time in between. Assessed with r or Kappa. Scores closer to 1 = better. High positive scores means that how you responded one time is highly correlated with how you answered a second time. .97 is great!
Explain face validity, content validity, criterion validity, convergent validity, and discriminant validity.
NON-EMPIRICAL WAYS:
- Face validity: degree to which a scale appears to measure the intended construct. Does a scale on face value seem to measure what you’re trying to measure? This is all about appearance, does it look like questions related to the construct you’re trying to measure? Not empirical because just looking at it isn’t a good measure.
- Content validity: degree to which the item on a scale reflect the range of material that should be included in measurement of the construct. Does the scale contain what you think it should contain? Are all the key components from the construct included? Is a scale including all of the material related to that construct?
EMPIRICAL WAYS
- Convergent validity: measure is associated with other measures of theoretically similar constructs. Ex: satisfaction with life scale and similar measures. Is this measure associated with other similar constructs? Ex: shyness and social anxiety – they are different things but should be related to each other. Can be both negatively and positively correlated.
- Discriminant validity: Measure is not associated with measures of other, theoretically different constructs, Ex: satisfaction with life scale and social desirability. The opposite of convergent. Some concepts are not going to be related to one another, so we want to see if our measure is distinct from those other measures. We shouldn’t see a correlation between our measure and some unrelated topic. Ex: life satisfaction should be unrelated to social desirability.
- Criterion validity: measure is associated with behavior or concrete outcome that it should be associated with. Ex: satisfaction with life scale and diagnoses of physical or mental health disorders. We’re interesting in if this measure relate to some kind of concrete outcome it should be associated with.
Be able to interpret statistics about correlations.
Correlations: how A and B are linearly related to each other. Ex: alertness and how many hours someone slept last night.
- R (correlation coefficient) ranges from -1 to 1
- Sign indicate relationship
Positive: as one variable increases, the other does too. High/low scores for one variable is associated with high/low scores for the other. Variables are moving in the same direction.
Negative: as one variable increases, the other variable decreases. High scores on one variable are associated with low scores on the other.
No association: variables are not related.
- Absolute value indicates magnitude/strength. Don’t pay attention to the +/- sign – it just indicated positive/negative correlation and has nothing to do with strength. When data points are spread out = not a strong relationship. When they are closer together = stronger relationship.
.1/-.1 is weak
.3/-.3 is moderate
.5/-.5 is strong
Differentiate between measured and manipulated variables.
- Measured variables:
- variables you observe or record
- Ex: height, marital status, extraversion
- Manipulated variables:
- are controlled by experimenter
- ex: participants randomly assigned to receive candy before exam or no candy. Exams scores are measured, manipulated whether given candy or not.
- Measured or manipulated?
- age: measured
- number of siblings: measured
- dose of drug given to ppt, 1 mg or 5 mg: manipulated with two levels
Know the benefits of experimentation.
- Allows us to determine causality: one variable causes changes in another variable
Explain features of experiments.
- Covariance: are different levels of the IV associated with different scores on the DV? Ex: exam scores and candy. IV is candy (two levels; yes or no), DV is exam scores. So are we going to have different exam scores based on whether people got candy or no candy. Basically are the scores on our DV going to vary based on the level of the IV?
- Temporal precedence: the IV is manipulated before the DV is measured. Cause comes before effect. So give participants candy or not before they take the exam, not after the exam.
- Internal validity: we need to be able to determine that there is no other kind of good alternative explanation for the change in the DV other than what we did to the IV. A well-designed experiment can meet this rule. Biggest threat is lack of random assignment.
- Random assignment: method of placing participants that ensures participants have an equal chance of being in any group. We don’t want differences between our groups to start with. We don’t want students with 4.0 GPA in one group.
Identify an IV and DV.
- Independent variable (IV): variable that is manipulated. Each level is called a condition.
- Dependent variable (DV) variable that is measured
Differentiate between experimental and control groups.
- Experimental group: the group that gets the key treatment in an experiment
- Control group: group that serves as a comparison group in an experiment
- Types of control groups:
- empty control group: a group that does not receive any form of the treatment and just completes the dependent variable. Ex: getting no candy.
- placebo group: a group where participants believe they are getting the treatment, but in reality they aren’t. Ex: thinking they’re getting candy but they are not.
Identify confounds.
any variable that the researcher unintentionally varies/manipulate along with the manipulation. Can lead us to draw wrong conclusions. Everything could be exactly the same except for the IV.
Describe the research process for experiments.
Steps in experimental research:
1) Write null and alternative hypothesis
2) Select participants
How many participants do you need? Make sure you got a sufficient sample size to make sure you got sufficient power.
Run a power analysis
3) Conduct experiment
4) Analyze data
5) Make conclusions about your null hypothesis
6) Share findings
Null hypothesis: no difference between groups on dependent variable. H0. Reject null hypothesis means that there are differences between groups.
Alternative hypothesis: difference between groups on dependent variable. Ha
Independent group design vs. Within-subjects design
independent group designs; different groups of participants are placed into different levels of the independent variable (also called between-subjects or between-groups design). People are in one group or the other.
Within-subjects design: there is only one group of participants, and each person is presented with all levels of the independent variable (also called within-groups design). They are in all the different groups.
Types of within-subject designs:
- Pretest/posttest design: participants are measured twice, once at the beginning of the study and again at the end of the study.
- Longitudinal research: participants are repeatedly measured on the dependent variable over a period of time.
- Repeated measures design: participants are measured on the dependent variable after exposure to each level of the independent variable and they are recorded multiple times.
Advantages if within-subject designs:
- Appropriate for answering certain types of research questions: ex: if you want to measure change of attitudes/behaviors etc., within-subject design is better at looking at change than a between-subject design. Also good when looking at relative comparison: ex. Looking at people’s level of aggression when they have not been sleep deprived and then same people being sleep deprived for 24 hours and measure their levels of aggression again.
- Need fewer participants: you’re getting twice the data from the same number of participants. Ex: if you got 100 participants, you’re getting 200 data because they are all going to be in both groups.
- Avoid problems of individual differences in comparison groups: we don’t want our groups to start off with any kind of preexisting differences so we avoid that with within-subject designs because they are the same people.
