evaluating quantitative research

Question 1

reliable instrument

Answer 1

instrument that gives the same measurements when you repeatedly measure the same unchanged objects or events
-a test must be reliable for it to be valid

Question 2

internal validity

Answer 2

the extent to which the change in the dependant variable can be attributed to the manipulation of the independant variable

Question 3

external validity

Answer 3

extent to which researchers can generalize their findings to other people situations and times
-people : pop validity (do findings generalize from kids to adults)
-situations : ecological validity (real world vs gym situations)
-times : time periods, diff attitudes, beliefs

Question 4

internal vs external validity

Answer 4

trade-off between internal and external validity
-series of experiments (studies)
-each study has specific goals and limitations
-real world intervention vs. lab based intervention
->control for extraneous variables -> looks less like real world because controlling too much

Question 5

threats to internal validity

Answer 5

experimental procedures, treatments, or experiences of the participants that threaten the researchers ability to draw correct inference from the data about the population in an expriment
-ability to establish cause and effect

Question 6

history (threat)

Answer 6

events occuring during the course of experiment that cause changes in the DV and are not the intervention (IV)
-unplanned event that coincides with the IV could be responsible for the observed changes in DV

pretest -> unplanned event -> posttest

Question 7

maturation

Answer 7

processes within the participant that operate as a result of time passing
eg. developing in highschool during start of year beep to end of year beep

pretest-natural development -> post test

Question 8

testing

Answer 8

the effects of one test on subsequent administrations of another test
-skipping intervention? pre test right to post

Question 9

testing - reactivity

Answer 9

exposure to pretest changes behaviour
eg. self monitoring - athlete increase effort bc they were asked to record it
eg. food log

Question 10

testing - warm-up effect

Answer 10

do better on the subsequent measure/test because they are more familiar with it
-eg. getting better at yoyo test, not improving aerobic capacity just more familiar with test

Question 11

loss of naivety

Answer 11

people start catching on to what you are testing/measuring

Question 12

instrumentation

Answer 12

quantitative research relies on measurement
eg. leadership style, MRI , motivation etc.

instrumentation : changes in instrument calibration, including lack of agreement within and between observers
-it is important that the data acquired from instruments be accurate or else the interpretation will be inaccurate

-getting accurate measurements

Question 13

selection bias

Answer 13

choosing comparison groups in a non-random manner (not randomly assigned)
-pre-existing differences between groups - functions as confounding variables

Question 14

experimental mortality

Answer 14

loss of participants from comparison groups for non-random reasons
-participants drop out and those who remain tend to be more motivated (results in higher scores on post-test)

Question 15

selection - maturation interaction

Answer 15

the passage of time affects one group but not the other in nonequivalent group designs

-2 groups
group 1 has more enjoyment than 2
-but; could be due to the fact that

group 2 train on grass in worse weather
group 1 train on turf

Question 16

expectancy

Answer 16

researchers anticipating that certain participants will perform better
-expectancy has big impact on how we interact with people and how we receive them
-if we expect ppl to do well they will rise to the occasion

could do blinding procedures

Question 17

controlling threats to internal validity - randomization

Answer 17

random assignment - controls for history
matched pairs - eg. age, gender (next best thing)
randomizing treatments or counterbalancing
-> ABC v ACB c BAV v BCA (order of exercises in s and c)

Question 18

random selection vs random asignment

Answer 18

population -> (random selection) -> sample -> (random assignment) -> groups

random selection
-effects external validity
-ability to generalize findings

random assignment
-effects internal validity

Question 19

placebos, blind and double-blind studies

Answer 19

helps reduce experimenter bias/participant expectancies
-attention control group

blind : patients dont know what group they are in
double blind : patients AND researcher dont know what group patients are in

Question 20

standardize experiments and instruments as much as possible

Answer 20

eg. script every researcher uses, everything in the same order, make sure instruments are calibrated after every test
-order of questionnaires

Question 21

threats to external validity - two streams

Answer 21

population validity : extent to which findings from a sample can be generalized to a broader population

ecological validity : extent to which research emulates the real world

Question 22

time

Answer 22

a threat to external validity
-context changes over time, work needs to be contextualized for appropriate time
-covid studies generalized for that time only cant cut across time periods

Question 23

threats to external validity arise when

Answer 23

experimenters draw incorrect generalizations from the sample data to other persons, other settings, and past or future situations

Question 24

reactive/interactive effects of testing

Answer 24

the pretest may make the participant more aware or sensitive to the upcoming treatment
-as a result the treatment is not as effective without the pretest

eg. transformative leadership
-more receptive to leadership style because you have thought about your own style beforehand with PRETEST

Question 25

interaction of selection bias and the experimental treatment

Answer 25

when a group is selected on some characteristics, the treatment may work only on groups possessing that characteristic
eg. gender, age, sport, nationality

eg. gameplan for high perf sport
-helps folks while in sport ; wellbeing. may be effective in sport systems but less effective in other countries

Question 26

multiple treatment interference

Answer 26

when participants receive more than one treatment , the effects of the previous treatments may influence subsequent ones

eg. not gonna hangclean only for 3 months
-interaction bw clean and sprint
-could be less effective without one of them
-hard to generalize findings bc influencing each other

Question 27

controlling threats to external validity

Answer 27

selecting participants, treatments, experimental siutation and tests to represent a broader population
-random selection

Question 28

controlling threats to external validity

Answer 28

researcher might restrict claims about groups to which the results cannot be generalized
-conduct additional experiments with groups with different characteristics