Module 8, Hypothesis Testing I Flashcards
Hypothesis Testing:
statistical procedure for testing whether a research hypothesis provides a plausible explanation for experimental findings
Value of competing hypotheses
Two competing hypotheses are initially proposed; data is then gathered and statistical tests carried out determine which of the two hypotheses can be rejected or supported based on that data analysis
Considers the probability that the outcome of a study could have occurred in the absence of any effect of the experimental procedure or any difference between the populations being measured
Key Elements of an Experiment
Research hypothesis that summarizes the outcome to be tested
Null hypothesis that is complementary to the research hypothesis and summarizes the state of affairs, if the alternative hypothesis cannot be supported
Descriptions of the variables being tested, and an explanation of control measures for extraneous variables
Description of population of interest; how population is represented in the experiment
Identification of the experimental group and control group
Statement defining how the results are to be measured and interpreted
Null Hypothesis
H0: statement about a population parameter that is assumed to be true unless there is convincing evidence to the contrary, OPPOSITE OF ALTERNATIVE HYPOTHESIS
Hypothesis that our experimental group came from the population of normal responders; therefore it is representative and there was no effect, everyone might as well have been in the same group
Alternative/research Hypothesis
H1: statement that is directly contradictory to the null hypothesis
Hypothesis that our experimental group did NOT come from the population of normal responders/same population as the control group
Which hypothesis is the one being tested?
NULL HYPOTHESIS IS THE ONE BEING TESTED, NOT THE ALTERNATIVE: IF THE NULL HYPOTHESIS CAN BE REJECTED, THEN SUPPORT ACCRUES FOR THE ALTERNATIVE HYPOTHESIS
Directional Hypothesis:
states that one measure will be more or less than a comparison measure; SPECIFIES THE DIRECTION OF THE EXPECTED DIFFERENCE
NON-DIRECTIONAL HYPOTHESIS: STATES THAT TWO MEASURES WILL BE DIFFERENT FROM EACH OTHER, BUT DOESN’T SPECIFICY THE DIRECTION OF THE DIFFERENCE (most real world research hypotheses are non-directional)
If alternative hypothesis is directional
If alternative hypothesis is directional, the null hypothesis should predict that the populations are not different IN THE EXACT WAy predicted by the alternative hypothesis
The Comparison Distribution
Statistical distribution to which the results of a study are to be compared
Represents the situation or the true state of affairs in the case where the null hypothesis is true
Cutoff Score
Critical value that marks certain areas of the comparison distribution used as a reference for tests of significance
A z score that cuts off a given proportion of the distribution’s scores in one of the tails of the distribution acts as a marker for that area
One Tailed:
an alternative hypothesis that predicts a more than outcome or a “less than” outcome, leads to a one tailed test
Since the predicted outcome is in a SPECIFIC direction, only a result in that direction can justify the rejection of the null hypothesis
One Tailed: Right-Tailed vs. Left-Tailed Test
PREDICTION OF HIGHER SCORE = RIGHT TAILED TEST
PREDICTION OF LOWER SCORE = LEFT TAILED TEST
Two-Tailed Test
A hypothesis that predicts a “different from” outcome and doesn’t specify a direction of the effect that leads to a two-tailed test, because an outcome in either direction can justify the rejection of the null
Rejection Region
Alpha sets the significance level of the test and determines the size of the rejection region
A test statistic, z-score or t-score, that exceeds the predetermined critical value provides evidence for rejecting the null hypothesis in favour of supporting the alternative hypothesis
Statistical vs. Practical Significance
Something can be statistically significant without being practically significant
Statistical Signifance: probability that the observed difference occured by chance less than 0.05; a set of measurements or observations in a study is said to be statistically significant if it is unlikely to have occurred by chance
Effect Size:
way of quantifying the size of the difference between two groups so that we can better judge the practical significance of the results
PROBLEM WITH ONLY USING P VALUES WITHOUT CONSIDERING EFFECT SIZE:
: SAMPLE SIZE DIRECTLY AFFECTS THE CALCULATION OF THE RESULTS
The larger the sample size = the larger the cal;culated value of the test statistic
Cohen’s D
Used when comparing mean scores of two groups
D of 1: tells us that the two groups means differ by ONE STANDARD DEVIATION
Effect size of 0 would place the mean of group 2 at the 50th percentile of group 1, so the distributions overlap completely
Type I vs. Type II Error
Type I Error: rejecting the null hypothesis when it is true; OUTCOME REPRESENTS A FALSE POSITIVE
TYpe II Error: say null hypothesis is true when its actually false; PROBABILITY OF A TYPE 2 ERROR INCREASES AS THE PROBABILITY OF A TYPE I ERROR DECREASES (INVERSRLY RELATED TO THE LEVEL OF SIGNFICANCE), REPRESENTED BY BETA
Power of a test
Power of a test is equal to the probability of maing a correct decision and rejecting the null hypothesis when it is false (PROBABILITY OF DETECTING AN EFFECT IF THERE IS ONE)
FACTORS IMPACTING POWER:
sample size (larger n=greater power), significance level, the higher the value of alpha the higher power of the test (increasing alpha has the effect of increasing the size of the rejection region and decreasing the size of the non-rejection region=greater likelihood of rejecting the null hypothesis), true value of the parameter being tested (the greater the diff between the true value of a parameter and that specific in the null hypothesis, the greater the power of the test)
Problem with increasing alpha to increase power?
Increases probability of making a Type I error
because alpha is the probability of a type I error
hy does less variability increase power
That is, less within-group variance will allow the treatment effect between groups in the numerator to be more apparent.
why does a further location of true parameter increase power?
The location of the true mean. The further it is away to the tested one, the easier it is to detect the difference and reject the H0 , which implies a higher power of a test; Significance level α .
