EBP2 Correlation and Reliability Flashcards by Kaycie Colbert

What kind of data does correlation and regression deal with?

continuous

How well did you know this?

Not at all

Perfectly

What is the question to ask during correlation?

strength of association

How well did you know this?

Not at all

Perfectly

What is the question to ask during regression?

strength of prediction

How well did you know this?

Not at all

Perfectly

Describe correlation study.

pairs of scores and whether they covary

how strong is their linear relationship? what is the nature of the relationship?

How well did you know this?

Not at all

Perfectly

Correlations quantify strength of what type of relationship?

linear

How well did you know this?

Not at all

Perfectly

What is used to quantify strength of correlation?

correlation coefficients

How well did you know this?

Not at all

Perfectly

What is the range of correlation coefficients?

0 and +/- 1.00

closer to 1.00 = higher strength of relationship
sign indicates direction
tighter grouping means higher coefficient

How well did you know this?

Not at all

Perfectly

An r of 0.00-0.25 generally signifies what kind of relationship?

little or no

How well did you know this?

Not at all

Perfectly

An r of 0.26-0.50 generally signifies what kind of relationship?

fair

How well did you know this?

Not at all

Perfectly

An r of 0.51-0.75 generally signifies what kind of relationship?

moderate to good

How well did you know this?

Not at all

Perfectly

An r of 0.75-1.00 generally signifies what kind of relationship?

good to excellent

How well did you know this?

Not at all

Perfectly

What is the coefficient of determination?

square of the correlation coefficient (r^2)

is the percent of variance in y that is explained (or accounted for) by x

(amount of overlap between two circles)

How well did you know this?

Not at all

Perfectly

Describe the null hypothesis and significance of coefficient.

null hypothesis: correlation between variables is not significantly different from zero

significance means likely not 0 in population but not strong

very sensitive to sample size

How well did you know this?

Not at all

Perfectly

What is the most common type of correlation coefficient and briefly describe it?

pearson product-moment correlation

both variables continuous (interval or ratio)

How well did you know this?

Not at all

Perfectly

What is the non-parametric analog of Pearson r and briefly describe it?

Spearman rank (rho) (rs)

1 continuous, 1 ordinal or 2 ordinal variables

How well did you know this?

Not at all

Perfectly

What type of correlation is used when one variable is dischotomous and the other is continuous (interval or ratio)?

point biserial correlation (rpb)

How well did you know this?

Not at all

Perfectly

True/False: Point biserial correlation has the same results as t-test.

true

t test will only give statistical significance where rpb also gives strength or relationship

How well did you know this?

Not at all

Perfectly

Describe Rank Biserial correlation (rrb).

one variable is dichotomous, other is ordinal

computationally about the same as Spearman Rank

Results same as Mann-Whitney U-Test

How well did you know this?

Not at all

Perfectly

What correlation is used when both variables are both dichotomous?

phi coefficient

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when both variables are continuous?

pearson r

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when one variable is continuous and one is ordinal?

spearman rho

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when both variables are ordinal?

spearman rho

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when one variable it continuous and one it nominal?

point biserial

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when one variable is ordinal and the other is nominal?

rank biserial

How well did you know this?

Not at all

Perfectly

What correlation coefficient is used when both variables are nominal?

phi coefficient

True/False: Correlation does not assess differences or agreement.

true ICCs do

True/False: Causation equals correlation,

False - both may be caused by another factor causation statements come from controlled experiments (RCTs)

True/False: Can't generalize beyond range of scores in sample.

true

What is reliability?

the extent to which a measurement is consistent and free from error a reliable measure can be expected to repeat the same score on two different occasions provided that the characteristic of interest doesn't change

What is used to quantify reliability?

reliaility coefficients

What are the reliability coefficients for continuous (interval/ratio) data?

Pearson correlation (r) *Intraclass correlation coefficient (ICC)

What are the reliability coefficients for discrete (ordinal/nominal) data?

Percent agreement *Kappa

What are the downfalls to pearson's r reliability coefficient?

assesses relationship, not agreement only two raters or occasions could be compared so they can have perfect correlation but not agree

What is the formula for reliability coefficients?

rxx = true score variability / (true score variability + error variability)

Score _ = no reliability, | Score _ = perfect reliability

0, 1

The _ the true score variability reduces reliability.

less

The _ error variability, reduces reliability.

Which of the following is not true for ICC? a. measures degree of relationship (association) and agreement b. can only be used with one rater or ratings c. designed for interval/ratio d. Often reported in conjuction with SEM

b - can be used with >2 raters or ratings

ICC gives _ estimate of reliability. | SEM gives _ estimate of reliability.

standardized(no units), unstandardized (in units)

What are the factors that ICC type depends on?

purpose of study, design of study, types of measurements taken

ICC type is defined by two numbers in parentheses. What position in parentheses describes the model? form?

first, second

What is a model 1 for ICC?

each subject measured by different set of raters randomly chosen

What is a model 2 for ICC?

each subject measured by same raters "randomly" chosen and representative of rater population results are generalizable most common for inter-rater reliability or test-retest reliability

What is a model 3 for ICC?

each subject measured by same rater(s) of interest so not generalizable most common for intra-rater reliability

What does the form in the ICC type signify?

number of observations used to obtain reliability estimate | 1 observation per subject rater = 1, 3=3, 4=4, etc

What does an ICC score of > 0.90 generally mean?

best for clinical measurements

What does an ICC score of >0.75 generally mean?

good reliability

What does an ICC score of < 0.75 generally mean?

poor- moderate reliability

ICC estimate us based on _ measures.

average

True/False: ICCs should always include a 95% CI.

true

In ICCs, P values tests whether point estimate is _.

statistically different from 0

Describe percent agreement

how often raters agree 0%-100% divide # of agreements by total of all possible agreements based on frequency table agreements on diagonal, disagreements are all others

True/False: Percent agreement account for agreement due to chance.

false - it does not it tends to overestimate reliability

Describe Kappa coefficient.

proportion of agreement between raters after chance agreement has been removed used on nominal and ordinal data

Describe weighted Kappa.

can choose to make penalty worse for larger disagreements weights can be arbitrary, and symmetric or asymmetric best for ordinal data

Generally, a kappa value of <0.4 indicates what about agreement beyond chance?

poor to fair

Generally, a kappa value of 0.4-0.6 indicates what about agreement beyond chance?

moderate

Generally, a kappa value of 0.6-0.8 indicates what about agreement beyond chance?

substantial

Generally, a kappa value of 0.8-1.0 indicates what about agreement beyond chance?

excellent

What is the most commonly applied statistical index for internal consistency?

cronbach's alpha

What is internal consistency?

reflection of the correlation among items and the correlation of each individual item with the total score

True/False: Cronbach's alpha can be used for dichotomous or ordinal (multiple choice) data.

true

What is the recommended range for Cronbach's alpha?

between 0.70 to 0.90 If too high, probably redundant. If too low, possibly measuring different traits

What is response stability?

stability of repeated measures over time ~test-rest reliability

What are the three commonly used statistical methods to express response stability?

Standard error of measurement, Minimal detectable difference/chance, coefficient of variation

What is the standard error of measurement?

measure of reliability using standard deviation of the measurement errors SD x square root of 1-ICC can be used to create a 95% CI around a measurement (95% CI = Score +- 1.96(SEM)

What is minimal detectable difference?

amount of change in a variable that must be achieved to reflect a true change/difference mathematical multiple of SEM MDC = mean +- (1.06)(SEM)(sqrt2)

What is coefficient of variation?

ratio of SD to mean, expressed as percentage CV = (SD/mean) x 100

What are alternate forms?

comparing different methods of testing same phenomenon with different instruments

What is Limits of Agreement?

range that includes ~ 95% of differences 95%LOA = X +- 2s

Describe a Bland-Altman plot.

spread of scores around a 0 point to help decide if the observed error is acceptable if we substitute one measurement method for another If all on one side of 0, consistently measuring more tighter range = more agreement

What question does correlation seek to answer?

Is there a strength in the association between variables?

True/False: You are reading a study that has shown a strong correlation between hours studying (greater than 3 hours) for an EBP II exam and getting a 90% or above (r = .94). We can now infer that studying for 3 or more hours will cause one to get a 90% or above on the exam.

False - correlation does not equal causation

ICCs are primarily used with dichotomous or categorical data.

continuous data (ratio/interval)

You are wanting to use a reliability process that is the most conservative approach. What would be most appropriate if this were your goal?

model 1 ICC

You are reading a study that is reporting ICC values and note that the following was reported "ICC(2,1)". What does this tell the reader?

Each subject measured by same raters using an inter-rater approach where only one observation per subject per rater was performed.

You are wanting to assess for the percentage of between raters who are performing the Lachman's test on those suspected of an ACL tear. The raters are rating their findings as either positive or negative. What would most appropriately assist with this?

Kappa

You come across a Kappa value of .54. How would you interpret the magnitude of this value?

moderate agreement

True/False: SEM is a measurement that takes into account ICC.

true