Evaluating Quantitatve Research

Question 1

Internal consistency

Answer 1

Consistency of scores within a test, or degree of interrelatedness among items (extent to which items on measure are measuring same thing)

Question 2

What is internal consistency assessed by

Answer 2

Cronbach’s coefficient alpha, ranging from 0-1
- 0 not related
- higher items covary the greater the internal consistency
- affected by number of items

Question 3

What are good and poor example of internal consistency

Answer 3

.85 good
.45 poor

Question 4

What is test-retest reliability

Answer 4

• method for estimating reliability
• coefficient between scores on repetitions of the same test
• if scores stable over time, more reliable

Question 5

Can a measure be reliable, yet not valid or valid and not reliable

Answer 5

Reliable, yet not valid

Question 6

Internal validity

Answer 6

Claim of researchers that any change in an outcome is a result of their intervention and not other factors

Question 7

What are internal validity threats?

Answer 7

Factors that may alter the dependent variable separate from the effects of the independent variable

Question 8

Mortality (experimental drop out)

Answer 8

People leaving study because of boredom, sickness, injury, inconvenience, discomfort, etc.

Question 9

Testing

Answer 9

Participants may be one familiar with test, and this might influence future performances

Question 10

Diffusion of treatments

Answer 10

If participants in experimental and control groups communicate with each other communication can impact scores
Blind study can fix

Question 11

Halo effect

Answer 11

Researchers have some expectation about performance of participants and are in position of assessment

Question 12

Maturation

Answer 12

Participants in study might mature naturally or change in many ways over time

Question 13

History

Answer 13

Events other than the experimental treatment can affect results

Question 14

Regression

Answer 14

Participants who have extreme score (high or low), scores naturally change in direction of mean

Question 15

Selection bias

Answer 15

When groups are formed without random assignment, possibility that they will be biased

Question 16

Placebo or Hawthorne effects

Answer 16

Participants may react in a way they expect they should when they are in experimental study (placebo)
Might react in favourable ways just because being observed (hawthorne)

Question 17

Strategies for controlling internal validity

Answer 17

Control group, random assignment, placebos, blind setups and double-blind setups

Question 18

External validity

Answer 18

• generalization of research findings to other people, contexts, or time

Question 19

Selection and treatment interaction

Answer 19

When unique characteristics of participants involved in study makes treatment effective only for them

Question 20

Setting and treatment interaction

Answer 20

When studies are conducted in highly controlled environments, it is challenging to know if same results would occur in real-world

Question 21

History and treatment interaction

Answer 21

Findings could not be generalized to past or future settings because results are dependent on timing of implementation