Statistical tests

Question 1

What tests do we use for normally distributed univariate analysis of two independent samples of quantitative variables?

Answer 1

Pooled (two sample) t test or Welch’s t test (no need for equal variance but normality). Both compare means.

pooled is same as outcome ~ treatment

If we have a balanced design (equal sample sizes in the two groups) then the pooled t approach and the Welch’s t approach yield essentially the same results

Question 1

What tests do we use for a univariate analyse of two independent samples of quantitative variables if we can’t assume normality?

Answer 1

• Bootstrap mean difference (compares means)
• Wilcoxon-Mann Whitney rank-sum test (no equal variance/normality, estimated median of difference)

Question 2

What are characteristics of a paired sample?

Answer 2

Every value is connected to a corresponding value for the same subject
Calculate the difference for each subject, so long as there aren’t any missing data

Question 3

What paired sample test do you use if normally distributed?

Answer 3

Paired t-test (one sample t test)

Question 4

What paired sample test do you use if not nomally distributed?

Answer 4

• Bootstrap population mean (Hmisc::smean.cl.boot)

Question 5

What test do you use for analysis of two or more means when normally distributed with equal variances?

Answer 5

Analysis of variance (ANOVA)

test is fairly robust to violations of the Normality assumption

and variance is less of an issue in a balanced design

Question 6

What test do you use for analysis of two or more means if difference in variance or skew?

Answer 6

• identify pairwise differences while correcting for multiple comparisons - Bonferroni or Tukey’s Honestly Significant Difference (HSD)
• severely skewed data - use Kruskal-Wallis test

Question 7

What tests are used to test association between categorical variables

Answer 7

Pearson chi-square and Fisher’s Exact
compare observed cell counts to what would be expected if variables were independent.

Question 8

What assumptions must hold true for Pearson chi-squared test?

Answer 8

• Assumes that the expected frequency will be at least 5 (and ideally 10) in each cell
• Cochrane conditions: no cells with zero counts and at least 80% of the cells in our table with expected counts of 5 or higher
• If assumptions not met then collapse categories
• If can’t collapse then just describe data without a statistical comparison

Question 9

When should we use Fisher’s exact test instead of Pearson squared test?

Answer 9

• When sample size is small
• expected cell count < 5
• often for 2x2 tables
• No assumptions about sample size

exact probability of getting table as extreme as observed if independent