Week 6 - Group Differences (Independent Samples t-test)

Question 1

What kinds of groups might be used in a research question?

Answer 1

1. Naturally occuring groups (eg. Males/Females)
2. Groups defined by research questions
3. Matched or Paired Sets of Measurements

Question 2

What are two types of matched groups?

Answer 2

1. Matched groups (2 twins, husband & wife)

2. Repeated measures for one group (ie. same individuals tested at time 1 and time 2)

Question 3

What is the fundamental difference between groups that are naturally occurring or defined by research questions and matched pairs groups?

Answer 3

1. In naturally occurring groups & those defined by research questions, group members are independent
2. In a matched/paired group, there is a dependency between the pairs.

Question 4

What is used to assess group differences between independent groups?

Answer 4

Group Means

Question 5

What is used to assess group differences between dependent groups?

Answer 5

difference between each pair of scores.

Question 6

What is the independent variable when comparing group differences?

Answer 6

The groups

Question 7

What is the dependent variable when comparing group differences?

Answer 7

The constructs being measured

Question 8

What is the independent t test used for?

Answer 8

To determine whether two independent groups come from the same population or different populations

Question 9

What group differences indicate that two groups come from different populations?

Answer 9

1. Different population means but same population variance
2. Different means and different population variance

Must be different means, but may have diff. variance too

Question 10

What is the Behrens-Fisher problem?

Answer 10

How to analyse data from groups with different population means and variances

Question 11

Define Standard Error as it relates to group differences?

Answer 11

The average variability in a sampling distribution of mean differences

Question 12

What is pooled variance?

Answer 12

A weighted average of the sample variances from both groups.

Question 13

What is the Standard Error or the mean difference?

Answer 13

the expected variabilitly in sample mean differences for two groups drawn from the same population having the same population variance.

Question 14

How is the SE of mean difference calculated?

Answer 14

Sm1 - m2 = square root of pooled variance divided by sample size for group 1 + the square root of the pooled variance divided by sample size for group 2

Question 15

What are the df for an independent t test?

Answer 15

df = n1 + n2 - 2

Question 16

How is the Tobs calculated for an independent t test?

Answer 16

sample mean difference - population mean difference, divided by the standard error of mean differences (calculated on sample values)

Question 17

What are the assumptions of the independent samples t Test?

Answer 17

1. Independence of observations
2. Populations are normally distributed
3. Homogeneity of Variance (pop’n variances are equal)

Question 18

What is a robust hypothesis test?

Answer 18

A hypothesis test in which the actual proportion of false rejections remain the same as the nominal value set by alpha when the statistical assumptions are not met (eg. alpha level of 0.05 results in an actual number of false rejections of 0.05)

Question 19

What is the nominal false rejection rate?

Answer 19

the number of false rejections set by the alpha level (eg. with alpha level of 0.05, we would expect 0.05 of all true null hypotheses to be rejected, this is the type 1 error rate)

Question 20

What is a liberal hypothesis test?

Answer 20

Actual number of false rejections is too large.

1. False rejection rate higher (larger) than alpha (the nominal level for false rejections)
2. Obtained p value is smaller than it would be if the false rejection rate was the same as alpha
3. More likely to reject true null hypothesis than the nominal level of false rejections set by the alpha level.

Question 21

What is a conservative hypothesis test?

Answer 21

Actual number of false rejections is too small.

1. Proportion of false rejections is less than the nominal value specified by alpha
2. The obtained p value is larger than it would be if the false rejection rate was the same as alpha
3. Less likely to reject true null hypothesis than the nominal level of false rejections set by the alpha level.

Question 22

Actual False rejection rate?

Answer 22

The proportion of times a true null hypothesis would be rejected in practice by researchers over repeated applications of same decision rule under a particular set of differing conditions.

Question 23

What is a robust confidence interval?

Answer 23

When the actual coverage of the interval remains the same as the nominal coverage set by the specified size of the interval when the statistical assumptions of the t-test are not being met.

Question 24

What is a conservative confidence interval?

Answer 24

One where the actual coverage of the CI is larger than the nominal 95% (interval wider than it should be)

Question 25

What is a liberal confidence interval?

Answer 25

One in which the coverage of the CI is smaller (narrower) than the nominal 95% coverage.

Question 26

What effect does unequal group sample size and heterogeneity of variance have on the t-test?

Answer 26

It is no longer robust and becomes either too liberal or conservative.

Question 27

When does a t-test become more conservative under violation of homogeneity of variance (with unequal sample sizes)?

Answer 27

When smaller sample size has smaller variance and larger sample has larger variance.

Question 28

When is the independent samples t-test robust to violations of homogeneity of variance and normality?

Answer 28

When sample sizes are the same and the distribution is normal (or only moderately non-normal).

Group variances can have mild-moderate departure from homogeneity of variance under these conditions.

Question 29

When is the independent samples t-test not robust?

Answer 29

When group variances are different (heterogeneous) and sample sizes are different.

Question 30

What adjustment is made to the t-test when homogeneity of variance is violated?

Answer 30

1. Standard error used calculated using separate variances (not pooled variance)
2. Degrees of freedom are adjusted to make the actual false rejection rate closer to the nominal value set by alpha.

Question 31

What is Levene’s Test used for?

Answer 31

To test for equality of variance between groups.

Question 32

What are the features of Levene’s Test?

Answer 32

1. F distributed

2. Null hypothesis assumes equal variance

Question 33

What does a small p value for a Levene’s Test mean?

Answer 33

We reject the null hypothesis that there is equal variance and use the t value for “ equal variance not assumed”

Question 34

What does a p value > 0.05 mean for a Levene’s Test?

Answer 34

That we assume that the homogeneity of variance is not violated if the

Question 35

When should a Levene’s test be used?

Answer 35

When the sample sizes are not equal.

Question 36

What are the features of Levene’s Test?

Answer 36

1. F distributed

2. Null hypothesis assumes equal variance

Question 37

What does a small p value for a Levene’s Test mean?

Answer 37

We reject the null hypothesis that there is equal variance and use the t value for “ equal variance not assumed”

Question 38

What does a p value > 0.05 mean for a Levene’s Test?

Answer 38

That we assume that the homogeneity of variance is not violated if the

Question 39

When should a Levene’s test be used?

Answer 39

When the sample sizes are not equal.

Question 40

In addition to an independent samples t-test, what else can be used to make inferences about the population?

Answer 40

A measure of effect size and its confindence interval

Question 41

What is a measure of effect side in an intependent groups design?

Answer 41

The standardised mean difference

Question 42

Name two measures of effect size (two types of standardised mean difference)

Answer 42

Hedges g and Cohens d

Question 43

What can a confidence interval generated using raw mean differences (rather than standardised mean differences)?

Answer 43

Whether the CI contains a value of zero or not

Question 44

What are cohen’s d and hedges g estimators of?

Answer 44

Standard mean difference (each uses different standardiser, but value of both will be effectively same when sample size >100)

Question 45

What is a standardised?

Answer 45

A standard deviation unit common to two groups in an independent measures design.

Question 46

What is the difference between the standardiser in hedges g and cohen’s d?

Answer 46

Hedges g used the df in the denominator of its standardised, where cohen’s d uses total sample size (n1 + n2)

Question 47

What happens to a CI produced around hedges g or cohen’s d when homogenaety of variance is violated?

Answer 47

The CI wil not be robust

Question 48

What is g*?

Answer 48

An unbiased estimation of Hedges g

Question 49

Is hedges g a biased estimator?

Answer 49

yes, but only for small sample sizes because also consistent.

Question 50

What is bonett’s delta?

Answer 50

An estimator of standardised mean differnece that is robust to violations of homogeneity of variance.

Question 51

When does a t-test become more liberal under violation of homogeneity of variance (with unequal sample sizes)?

Answer 51

When smaller sample size has larger variance and larger sample has smaller variance.

Question 52

What is g*?

Answer 52

An unbiased estimator of hedges g (when a correction factor has been applied)

g* = g x J(m)

Question 53

What value can the correction factor for g* have?

Answer 53

J(m) will always be less than 1

Question 54

What happens to g* when degrees of freedom are greater than 120?

Answer 54

The correction factor is almost one and hedges g becomes virtually unbiased.

Question 55

What effect does sample size have on the precision of a CI for hedges g ?

Answer 55

The larger the total sample size, the more precise the interval. Equal group sample sizes will further enhance the precision of the CI.