Chapter 10- Rigor and Validity in Quantitative Research

Question 1

Validity

Answer 1

“the appropriate truth of an inference”

inferences that a cause results in a hypothesized effect are valid to the extent that researchers can marshal strong supporting evidence

Validity is always a matter of degree, not an absolute

Validity is the property of an inference- not of a research design, but design elements profoundly affect the inferences that can be made.

Question 2

Threats to validity

Answer 2

reasons that an inference could be wrong

when researchers introduce design features to minimize potential threats, the validity of the inference about relationships under study is strengthened.

Question 3

statistical conclusion validity

Answer 3

concerns the validity of inferences that there truly is an empirical relationship or correlation, between the presumed cause and effect.

The researcher’s job is to provide strong evidence that an observed relationship is real.

Question 4

internal validity

Answer 4

concerns the validity of inferences, that, given that an empirical relationship exists, it is the independent variable, rather than something else, that caused the outcome.

Reserachers must develop strategies to rule out the plausability that some factor other than the independent variable accounts for the observed relationship

Question 5

construct validity

Answer 5

involves the validity of inferences “from the observed persons, settings, and cause and effects operations included in the study to the constructs that this instance might represent:

one aspect concerns the degree to which an intervention is a good representation of the underlying construct that was theorized as having the potential to cause beneficial outcomes.

another issue concerns whether the measures of the outcomes are good operationalizations of the construct for which it is intended.

Question 6

external validity

Answer 6

concerns whether inferences about observed relationships will hold over variations in persons, setting, or time.

relates to the generalizability of the inferences- a critical concern for evidence-based nursing practice

Question 7

Controlling Confounding Participant characteristics

Answer 7

1) Randomization
2) Crossover- participants serve as their own controls
3) Homogeneity-participants are homogenous with respect to confounding variables. results can not be generalized to tpye of people who did not participate in the study
4) Stratification/Blocking
5) Matching
6) Statisitical control

Question 8

Statistical Conclusion Validity

Answer 8

statistical methods are used to support inferences about whether relationships exist

Researchers can make design decisions that protect against reading false statistical conclusions

Question 9

Low statistical Power

Answer 9

statistical power-

When small sample sizes are used, statistical power tends to be low and the analyses may fail o show that the IV and DV are related- even when they are.

maximizing precision- which is achieved through accurate measuring tools, controls over confounding variables, and powerful statistical methods.

Question 10

Internal Validity

Answer 10

The extent to which it is possible to make an inference that the IV, rather than another factor, truly had the causal effect on the outcome (DV)

If researchers do not manage confounding variation, the conclusions that the outcome was caused by the independent variable is open to challenge

6 threats to internal validity- each threat represent an alternative explanation that competes with the independent variable as the cause of the outcome.

Question 11

Temporal ambiguity

Answer 11

the cause most precede the effect

establishing temporal sequencing may be difficult in correlational studies- it may be unclear whether the IV preceded the DV, or vice versa.

This is especially true in cross-sectional studies

Question 12

Selection threat

Answer 12

emcompasses biases resulting from preexisting differences between groups.

When not selected at randon, the groups copared are seldom completed equivalent

Selection bias is the most problematic and frequent threat to internal valididty in studies not using an experimental design

Question 13

History threat

Answer 13

Concerns the occurrence of external events that take place concurrently with the IV and that can affect outcomes.

Question 14

Maturation threat

Answer 14

processes occurring during the study as a result of the passage of time rather than as a results of the IV

a one-group pretest-posttest design is highly suspective to this threat

refers to any change that occurs as a function of time

Question 15

Mortality/Attrition threat

Answer 15

Individuals dropping out of a stop

attrition bias can occur in single-group quasi-experiments if those dropping out of the study are biased subset that makes it look like a change in average values resulting from the treatment

prospective cohort study- may be differential attributions between groups being compared because of death, illness, relocation.

the longer the study, the greater the risk

Question 16

Testing and Instrumental threat

Answer 16

Testing- the effects of takign a pretest on the posttest performance.

sensitization more likely to occur when exposed to controversial or novel material in the pretest

instrumentation- bias reflects changes in measuring instruments or methods of measurements between two points of data collect.
Can occur even if the same measurement tool is used (data collector more experienced, participants bored)

Question 17

Construct validity

Answer 17

the first step in fostering construct validity is a careful explication of the treatment, outcomes, setting and population constructs of interest.

the next step is to select instances that match those constructs as closely as possible.

further cultivated when researchers assess the match between the exemplars and the constructs and the degree to which any “slippage” occurred

has most often been a concern to researchers in connection with the measurements of outcomes

it is just as important for the IV to be a strong instance of the construct of interest as it is for the measured outcome to have a strong correspondence to the outcome construct.

in nonexperimental research, researchers do not create and manipulate the hypothesized cause, so ensuring construct validity of the IV is often difficult

covers outcomes, treatments, persons and settings

there are constructs that require careful description and the selection of good exemplars that match those constructs

Question 18

Threats to construct validity

Answer 18

Reasons that inferences from a particular study exemplar to an abstract constuct could be erroneous.

Such a threat could occur if the operationalization of the construct fails to incorporate all teh relevant characteristics of the underying construct or if it includes extraneous content

Question 19

Reactivity to the study situation threat

Answer 19

Participants may behave in a particular manner because they are aware of their role in a study Hawthorne effect)

these perceptions become an unwanted part of the treatment construct under study.

Question 20

Researcher expectancies

Answer 20

A similar threat stems from researcher’s influence on participant responses through subtle communication about desired outcomes.

Question 21

Novelty effects

Answer 21

when a treatment is new, participants and research agents alike might alter their behavior.

Question 22

compensatory effects

Answer 22

in intervention studies, compensatory equalization can occur if health care staff or family members try to compensate for the control group members’ failure to receive a perceived beneficial treatment

Question 23

treatment diffusion or contamination

Answer 23

alternative treatment conditions can become blurred, which can impede good construct descriptions of the independent variable.

this may occur when participants in a control group condition receive services similar to those in the treatment condition.

or when those in the treatment group drop out, putting them in the control group.

Question 24

external validity

Answer 24

concerns the extent to which it can be inferred that relationships observed in a study hold true over variations in people, conditions, and settings.

emerged as a major concern in the EBP world in which there is an interest in generalizing evidence from tightly controlled research settings to real-world clinic practice settings

may ask: whether a relationship observed in a study can be generalized to a larger population.

REPRESENTATIVENESS of the participants used in the study

REPLICATION- multi-site studies are powerful because more confidence in the generalizability of the results can be attained if findings are replicated in several sites.

systematic reviews are a crucial aid to external validity precisely because they illuminate the consistency of results in studies replicated with different groups and settings.

Question 25

threats to external validity

Answer 25

threats to external validity concern ways in which relationships between variables might interact with or be moderated by variations in people, settings, time, and conditions.

Question 26

Interactions between relationships and people- external validity threat

Answer 26

an effect observed with certain types of people might not be observed with other types of people.

A common complaint about RCTs is that many people are excluded- not because they would not benefit from the treatment, but because they cannot provide needed research data (do not speak Engligh, cognitive impairment) or they would not allow the best test for the intervention (complex comorbidities)

Question 27

Quiz info

Answer 27

Content validity concerns the degree to which an instrument has an appropriate sample of items for the construct being measured.

Criterion-related validity examines the relationship between scores on an instrument and an external criterion.

Construct validity examines these questions: What is this instrument really measuring?

Does it validly measure the abstract concept of interest? Face validity refers to whether an instrument looks as though it is measuring the appropriate construct; while this is good, it is not the most important measure of validity.

Reliability and validity are interrelated but not the same.

Question 28

External validity threats

Answer 28

Rationale:Common threats to external validity are reactivity (how a subject reacts to being studied); 
experimenter (when a characteristic of the researcher influences the study result); 
and novelty (when the knowledge of what is being done is new somehow effects the outcome, 
either favorably or unfavorably). 

Maturation and attritions are threats to internal validity.