Ch 8: Validity Flashcards
(22 cards)
External Validity/Generalizability
Addresses whether the results of the study can be generalized to a broader population
ex: weight loss drug works great in laboratory but not in real world, means it lacks validity outside the study
Construct validity
How well a measurement represents the intended concept
ex: Depression test correlated with existing depression measures
Construct: Discriminant (Divergent) Validity
is established when a measure does not correlate with measures of unrelated constructs (what you are measuring is distinct and not measuring something else)
ex: A test for job satisfaction should have low correlation with a test for social desirability, proving it measures job satisfaction and not just an individual’s tendency to answer favorably.
Construct: Convergent Validity
demonstrated when a measure is highly correlated with other measures that assess the same or similar constructs
ex: if you are studying how people feel about the Democratic candidate in an election, a good way to validate the scale would be to compare feelings toward the Republican candidate. If the two are negatively related, that would be a good sign that your measures work
Content Validity
refers to whether a measure fully represents the construct it aims to measure
ex: an exam must cover all relevant topics and not focus too narrowly on one topic (Group cohesion scale: multiple statements for each group [ATG-S])
Interparticipant bias
refers to when participants awareness of a study’s purpose influences their behavior
Ex: Participants who know they are being observed for stress might alter their behavior
Validity
the degree to which a measure or study accurately reflects the true value of what is being measured
have to have reliability (consistency) before validity
Internal Validity
concerned with whether the relationships observed in a study are accurate and genuine
Ex: a study measuring the effect of study habits on exam scores must control for other variables like prior knowledge (have you taken a class like this before?)
Ecological Validity
when the enviornment is as similar to real world as possible
Ex; lab participants asked to role-play employees –not much of it
Whereas actual employees put into the room would have more but not completely
Criterion Validity
Can I take one measure that allows me to predict some outcome
Ex: SAT scores predicted college GPA
Criterion: Concurrent Validity
correlates well with an existing measure taken at the same time
ex: (a new type of IQ test should produce similar results as an established IQ test if taken simultaneously)
Criterion: Predictive Validity
Can accurately forecast future outcomes
ex: (job aptitude test that accurately predicts future job performance has high [blank] validity)
Observer Effect (hawthorne)
presence of observer can influence participants behavior
Ex: workers might alter their behavior when they know they are being observed (Hawthorne studies)
The story goes that the researchers realized that the participants weren’t necessarily reacting to the specific changes, such as increased lighting. Instead, they were simply responding to the extra attention and scrutiny provided by the researchers.
History
historical events that occur during study can alter behavior and skew results
Ex: Study on stress may be affected by a major natural disaster occuring during the research period
Maturation
occurs when certain groups are over/under represented in a study
Ex: only surveying people who are already interested in the topic could lead to biased results
Morality
Any loss of participants during the study
ex: studying citizens opinions before and after the election. Post election they might be excited about a certain candidate, but after (their candidate lost) they may refuse to take part of the second part of study
lengthy studies more likely to be exposed to this harm
Regression toward the mean
In general, individuals who report extremely high or low scores on a scale at one point will later report a score that is closer to the average.
ex: A teacher may think that a video game is a great way to convey some important concept and assign those scoring at the bottom of the class to play the game as a way to improve their scores. But people with the worst scores in the class were likely to improve anyway. This doesn’t mean they will suddenly earn A’s, but the odds are that by the next exam, these same individuals will have surpassed some others in the course, regardless of the educational game.
Ceiling Effects
describes a situation in which initial scores are already high, making it unlikely they can increase further.
ex: comparing individuals with low and high concerns of diet, using eating behaviors as the dependent variable .
–>The low-concern individuals couldn’t do much to further reduce their intake, so any effects of media on them would be masked.
Floor Effects
describes a situation in which initial scores are already low, making it unlikely they can be decreased any further.
ex: comparing individuals with low and high concerns of diet, using eating behaviors as the dependent variable .
–> High-concern individuals may actually be less affected, but because they actually show a bigger decrease in eating, it would appear as if the content had as strong an effect, if not stronger, on this group.
Researcher bias
Any situation in which the actions or attitudes of the researcher distort the results
Ex: researcher might unknowingly favor certain data points that align with their hypothesis
solution: double-blind study or use scripts
Selection Bias
occurs when certain groups are over/under represented in a study
Ex: if you are conducting a study on body image and advertise it as such, you might not draw people who feel uncomfortable with their bodies
Sensitization
refers to when participants awareness of a study’s purpose influences their behavior
Ex: Participants who know they are being observed for stress might alter their behavior
suppose a study is focused on how individuals respond to conversations with people of different races. Further suppose that the researchers give an initial questionnaire that asks participants about their attitudes toward and exposure to people of different races. A participant might realize the study is about race and naturally decide that he or she should be careful to respond more favorably when interacting with someone of a different race.
