Research Design and Statistics Flashcards

Question

Threats to External Validity: Interaction between Testing & Treatment

Answer 1

**What is it?** pre-tests may sensitize the subjects to the purpose of the research study **AKA:** Pretest sensitization **Example:** pre-test before a film designed to reduce racism. The group who viewed the film may be primed and more motivated to pay attention to the film, as opposed to those who may watch the film without a pretest

Answer 2

**What are they?** cues in the research setting that may tip subjects off to the hypothesis People pleasers may act in ways to confirm the hypothesis, while others may act to disprove it

Answer 3

**What is it?** research subjects may behave differently simply because they are participating in research

Answer 4

**What is it?** DV is impacted by other aspects of the study **Example:** subjects get three treatments, always in the same order. Last treatment may show the best results, but there's no way of knowing if it's just from that treatment, or from impacts of the previous two

Answer 5

Take a random sample from subgroups of a population **Example:** random sample of different age groups

Answer 6

The unit of sampling is a naturally occurring group of individuals **Example:** residents of a city

Answer 7

Behaviour is observed and recorded in its natural setting Reduces many external validity concerns, but has no internal validity

Answer 8

Results of lab studies are used to draw conclusions about real-world phenomenon E.g. Milgram's obedience studies

Answer 9

**Single Blind:** subjects don't know what group they are in **Double Blind:** neither subjects or research know what group they are in Reduce demand characteristics, researcher bias and hawthorne effect

Answer 10

Controls for order effects by ensuring variables are received in different order **Latin Square Design:** order the administration of variables so that each appears only once in each position

Answer 11

Subjects randomly assigned to groups Groups receive different levels of manipulated variable Greatest for internal validity

Answer 12

**When to use?** when random assignment is not possible **Example:** studying a learning program that is being introduced to all grade 1 classes Next best for internal validity

Answer 13

**Internal Validity?** correlational research has none **Use for?** Prediction, esp. for variables that can't be manipulated

Answer 14

**Goal: **Assessing variables over time **Longitudinal:** same people studied over long time * *Pitfall:* underestimate changes, bc its often those who drop out that have the most significant changes **Cross-Sectional:** different groups of subjects, divided by age, are assessed at same time * *Pitfall:* cohort effects lead to overestimation of differences (e.g. may not account for an aid a different generation had, which is responsible for helping memory) **Cross-Sequential:** combines the two. Samples of diff groups are assessed more than once

Answer 15

Take multiple measurements over time (e.g. multiple pretest/posttest) to assess effects of IV **Benefits:** controls for threats to internal validity. You can add a control group to help with history effects **Example:** smoking reduction program in school. degree of post test results can indicate if it was a confounding factor or a result of the program

Answer 16

Can be one subject, or multiple that are treated as one group **Used for:** behaviour modification research Dependent variable measured multiple times during phases of the study (phase 1-no treatment/phase 2-treatment)

Answer 17

Single baseline and single treatment phase Phase 1: collect data on frequency of behaviour before treatment Phase 2: give treatment, collect data on if it reduced behaviour

Answer 18

**Benefits:** controls for extraneous factors, which AB does not **What does it do?** give treatment, withdraw treatment and reassess, and then provide treatment again. If behaviour continues again without treatment, the effect was likely due to treatment **Types:** *ABA*: baseline -> treatment -> withdraw *ABAB*: baseline -> treatment -> withdraw -> treatment

Answer 19

**Used when:** reversal not possible for ethical reasons It doesn't involve withdrawal of treatment Treatment applied sequentially **Multiple Baseline Across Behaviours:** start with one behaviour, then use same treatment for another **Multiple Baseline Across Settings:** home, school **Multiple Baseline Across Subjects:** try treatment on another subject

Answer 20

**Cons:** many threats to validity **Pros:** can try to ensure random sample **Types:** personal interviews, telephone surveys, mail surveys

Answer 21

**Con:** lack internal and external validity **Pro:** thorough on one person Useful as pilot studies that can ID variables to be studied in a more systematic manner

Answer 22

**What is it?** research involving the collection and analysis of verbatim reports **Example:** subject thinks aloud while doing something, which is then analyzed to look for themes/concepts evident as the subject performed the task

Answer 23

Unordered categories, none of which are higher than the others E.g. male/female

Answer 24

Provides info about the ordering of categories, but not specifics E.g. agree, strongly agree, neutral, etc.

Answer 25

Numbers are scaled at equal distances, but the scale has no absolute zero point e.g. IQ scores, temperature Multiplication or division not possible, but addition and subtraction are

Answer 26

Identical to interval, but they have an absolute zero E.g. dollar amounts, time, distance, height, weight, frequency of behaviours per hr

Answer 27

A summary of a set of data tables, bar graphs, histograms

Answer 28

Symmetrical, half scores above mean and half below Most scores are close to mean

Answer 29

May happen with ceiling/floor effects **Negatively Skewed:** has a tail on the left. Indicates *easy test* **Positively Skewed:** has a tail on the right. Indicates *difficult test*

Answer 30

Arithmetic average Add all values and divide by n **Con:** sensitive to extreme values

Answer 31

**What is it?** The middle value of data when ordered from lowest to highest (Md) **Odd groups:** literally the middle number **Even groups:** mean of the two middle numbers **Pros:** not as affected by extreme scores, so good for skewed distributions

Answer 32

**What is it?** the most frequent value in a set of numbers May have multiple modes (bimodal/multimodal)

Answer 33

**Normal Distribution:** all equal **Positively Skewed Distribution:** mean higher than median, median higher than mode **Negatively Skewed Distribution:** mean is less than median, mode is more than median

Answer 34

**What is it?** the difference between the highest and lowest scores **Cons:** impacted by extremes, so doesn't give accurate representation of the distribution

Answer 35

**What is it?** The average of the squared differences of each observation from the mean *For me*: Get the mean. How far is each score from the mean? Square that distance, and then add them all up. Take an average of the sum. This is variance. **What to know?** 1. measure of variability of distribution 2. many stat tests use it in formulas 3. It's equal to the square of the SD

Answer 36

**What is it?** the expected deviation from the mean of a score chosen at random Higher SD = more scores are likely to deviate from the mean

Answer 37

**What are they?** raw scores stated in standard deviation terms. Measures how many SD's a raw score is from the mean **Calculate by:** subtract teh sample mean by the score, and divide by the SD **Pro:** can compare across different measures and tests

Answer 38

**What are they?** mean of 50, Sd of 10

Answer 39

**Shape:**It is flat/rectangular. **Means:** within a given number of percentile ranks, there will always be the same number of scores

Answer 40

1. In a normal distribution, 68% of scores fall between -1.0Z and +1.0Z 2. In a ND, 95% of scores fall between z scores -2.0 and +2.0 3. In a ND, z-score +1.0 is a percentile rank of 84 (top 16%). -1.0 z-score is a PR of 16 (bottom 16%) 4. In an ND, z-score of +2.0 is 98th PR (top 2%). z-score -2.0 is PR of 2 (bottom 2%)

Answer 41

Most are around the mean (PR 50-84) At the extreme end there are less (84-98)

Answer 42

To allow us to make inferences about the population based on a sample

Answer 43

The inevitable error between the sample scores and the population

Answer 44

The extent to which a sample mean can be expected to deviate from its corresponding population mean

Answer 45

As sample size increases, the standard error decreases INVERSE relationship

Answer 46

**Null:** no difference between means of sampled populations. IV has no effect on DV. **Alternative:** IV does have an effect on DV

Answer 47

1. Retain null, no difference exists in population (**correctly retained**) 2. False null rejected, differences do exist in population (**correctly rejected**) 3. Null rejected, no differences exist (**incorrectly rejected**) 4. False null retained, differences do exist (**incorrectly retained**)

Answer 48

**One-tailed:** we hypothesize a particular direction. E.g. we anticipate one mean to be significantly higher than the other mean **Two-tailed:** hypothesize a difference in means, but not in what direction.

Answer 49

Null hypothesis is rejected but it is true | You think you have something but you really don't

Answer 50

Set by the research in advance, and it is the probability of making a Type I Error p = 0.05 or .01

Answer 51

Fail to reject null hypothesis, but it is false | Thinking you don't have something when you really do

Answer 52

**Power:** the probability of NOT making a Type II error * 1-beta * Sensitivity of a statistical test to detect an existing difference

Answer 53

1. Sample size 2. Alpha: higher alpha level = higher power 3. One-tailed tests are more powerful 4. Magnitude of Population Difference: more difference between population means = more likely to detect them. Can impact this by increases difference levels of IV

Answer 54

**Used for:** interval and ratio data **Assumptions:** 1.* Normal distribution of DV*. *Robust* 2.* Homogeneity of Variance*: variance of groups is equal. *Robust* 3. *Independence of Observations*: scores within same sample or group shouldn't be correlated (if they are, it means the scores could be impacted by a group factor) *Not robust*

Answer 55

**Used for:** DV measured on ordinal or nominal scale **Differences from Parametric:** * don't assume normal distribution * Less powerful **Similarity to Parametric:** * assume data come from unbiased sample **Types:** * chi-square * Mann-Whitney U

Answer 56

The obtained stat value is compared to a critical value in table 1. depends on the pre-set alpha level 2. degrees of freedom for the test

Answer 57

**Used for: **To test hypotheses about 2 different means *It cannot be used for more than 2 means* **Means:** t-ratio, if significant, indicates that the means are different

Answer 58

When a study involves only one sample Compare one mean to a known population mean Rarely used **degrees of freedom**: N - 1

Answer 59

**Used for:** compare 2 means from unrelated samples (e.g. treatment & control group; test scores of students from different schools; avg height of men & women) **Degrees of Freedom:** N - 2 **Assumptions:** -Homogeneity of variances -Data in each group ~normally distributed

Answer 60

**Used for:** samples that are related to each other somehow (e.g. matched sample, pretest-posttest) **Degrees of Freedom:** N-1 (N is the pair of scores)

Answer 61

* In a study w/ one independent variable where the means of more than two groups are compared * *What is the probability that these means are from the same population?* * F Ratio: if significant, the null is rejected * It doesn't tell you which means are different, so must do post-hoc tests

Answer 62

It represents a comparison between 2 estimates of variance 1. Between-group variance 2. Within-group variance

Answer 63

If the null hypothesis is true, the 2 estimates of varaince should be the ~same If null hypothesis false, the between-group variance should be HIGHER than within-group Differences between group means should be large enough to not be accounted for by error

Answer 64

variance between groups/variance within groups If top is BIG and bottom SMALL, that means it's significant

Answer 65

**What does it do?** measure of the variability of a set of data **In ANOVA Summary Table:** 1. between-group sum of squares 2. within-group sum of squares (error/residual) 3. Total Sum of Squares Used to calculate the F-Ratio

Answer 66

**Two Types** 1. df between (k - 1) 2. df within (N - k) *K = number of groups *N = total number of observations

Answer 67

**What is it?** the stat measure to estimate between and within-group variance **Mean Square Between**: sum of squares between / df between **Mean Square Within**: sum of squares within / df within

Answer 68

**Equation:** mean square between / mean square within

Answer 69

They make pairwise comparisons or complex comparisons between means **Pair wise:** compare means of novel treatment group to mean of typical treatment group **Complex:** compare combined mean of novel treatment and typical treatment with the control group mean

Answer 70

Increases risk of Type I error

Answer 71

1. Scheffe test is most conservative (most protection against Type I error, but increases chances of Type II) 2.If only doing pairwise comparisons, Tukey is the best one to choose

Answer 72

Used when all subjects receive all levels of the IV (e.g. group receives novel treatment and typical treatment)

Answer 73

When you need to adjust dependent variable scores to control for effects of extraneous variables

Answer 74

When study has more than one IV and you want to look at the effects of each IV separately (main effect) but also together (interactions) It helps you see the bigger picture, as the reality is that multiple factors play into dependent variables

Answer 75

the effect of one independent variable by itself

Answer 76

effects of an independent variable at the different levels of the other independent variables E.g. one-sided versus two-sided communication have different effectiveness based on a persons intelligence In graphs, can be seen as intersecting lines (e.g. in an X pattern)

Answer 77

**Mixed ANOVA:** more than one independent variable, but it has ~1 between subjects IV and ~1 repeated measured (within-subjects) variable

Answer 78

**When:** study involves 2+ dependent variables and 1+ independent variable. You want to look at the effect of each IV separately but also together **Why use it over multiple one-way ANOVA or factorial ANOVA?** Reduces the likelihood of Type I error

Answer 79

**Use for:** categorical data (nominal) e.g. survey results **Means:** the obtained frequencies in a set of categories differ significantly from null hypothesis

Answer 80

**Single sample chi-square:** C-1 **Multiple sample chi-square:** (C-1)(R-1) R = no. of rows

Answer 81

1. no observation can be related to one another. can't be used in before-after studies 2. each observation classified into only one category/cell (e.g. you can only belong to one political party) 3. Percentages of observations w/i categories can't be compared. Frequency data is required.

Answer 82

**Single Sample:** divide no. of subjects by the number of cells **Multiple Sample:**

Answer 83

**Used for:** * rank ordered ordinal data w/ two independent groups * Assumptions of independent t-test not met **Used when:** 1. when data from a research study are rank-ordered 2. 2 independent groups you want to compare 2. Assumptions of parametric tests are not met 3. Ordinal data (ranked but differences between ranks aren't consistent)

Answer 84

**Used for:** * Comparing two related groups (repeated measures) using rank-ordered data * Assumptions for paired t-test not met * Is there a consistent difference between the two sets of paired data? **Used when:** 1. assumptions of parametric are not met

Answer 85

* Comparing 3+ groups * Data not normally distributed; assumptions for ANOVA not met * Ordinal data * Question: Is there a significant difference in the ranks of the data between the groups?

Answer 86

* Used when calculating the relationship between two variables measured on an interval or ratio scale * Calculated based on z-scores, but don't need to know specifics

Answer 87

1. **Linearity:** assumes linear relationship between two variables, so can't be used for curvilinear relationships 2. **Homoscedasticity:** refers to an equal distribution of scores throughout the scattergram. Heteroscedasticity is when they are not equally dispersed. It lowers the r 3. **Range of Scores:** wider range makes for more accurate correlation

Answer 88

* The squared correlation coefficient * Indicates the percentage of variability in one measure (IV) that is accounted for by the variability in another measure (DV) * **E.g.** .70 correlation between IW and grades = 49% variation explained by IQ (get this by squaring the correlation coefficient)

Answer 89

Point-Biserial: * Look @ relationship between a continuous variable and dichotomous variable Biserial: * Look @ relationship between one continuous variable and an artifically dichotomized variable (a continuous variable that has been divided up)

Answer 90

**Phi Coefficient:** * 2 naturally binary dichotomous variables * No assumption about distribution **Tetrachoric Coefficient:** * Two artificially dichotomized variables * Assumes the variables are continuous and that they follow a normal distribution

Answer 91

* Correlation between two nominally scaled variables (unordered variables, each having more than two categories) * Describes how two categorical variables are related * Uses contingency tables * Things like Phi, Chi-square measure the strength of the associations between variables

Answer 92

* Correlation measure between two variables w/ an ordinal scale (ranked data) * Data not linear (monotonic), may have outliers * If data was linear and continuous w/o ranks, pearson r would be used * **E.g.** same students ranked on two different tests, Rho could be used to correlate them

Answer 93

* Strength of relationship between categorical and continuous variable * This measures NON-LINEAR relationships * Eta (n): tells you how much variance in the continuous variable is explained by the categorical variable * Eta (n2): expresses the PROPORTION of variance in the continuous variable that can be explained by the categorical variable * Often used with ANOVA * Ranges from 0 to 1

Answer 94

* An equation that is used to estimate the value of one variable based on the value of another * It finds the line of best fit, which is used to predict the dependent variable * **E.g**. can the EPPP score predict my performance ratings as a psychologist?

Answer 95

1. **Predictor/Independent Variable** 2. **Criterion/dependent Variable** Don't need to know the equation

Answer 96

1. **Linear Relationship** This is often depicted with the *line of best fit* (determined using least squares criterion) 2. **Error scores are normally distributed w/ a mean of 0** Independence of errors 3. **Homoscedasticity**: variance of residuals is constant across levels of IVs 4. **No perfect Multicollinearity**: IVs not highly correlated w/ each other 5. **Normality of Residuals:** 6. **Exogeneity:** IV's not correlated w/ error term 7. **Correct Model Specification:** includes relevant variables, excluded unnecessary ones

Answer 97

Code the subjects status on the IV using numbers, which are then put into the regression equation to predict a DV

Answer 98

* A measure of how well multiple IV's predict the DV in a multiple regression * Higher values = stronger relationship between combo of predictor variables at the criterion variable * R ranges from 0 to 1 * R2 (coefficient of determination) tells you the PROPORTION of variance in DV that is explained by IVs

Answer 99

The scores on more than one predictor are used to estimate scores on a criterion

Answer 100

1. Multiple correlation coefficient is highest when predictor variables have high correlations with criterion but low with each other (multicollinearity = they correlate) 2. Multiple correlation coefficient is never lower than the highest simply correlation between an individual predictor and the criterion 3. Multiple R can never be negative 4. Can be squared (coefficient of multiple determination)

Answer 101

When predictors have high correlations with one another in a multiple regression

Answer 102

* The multiple correlation squared * Indicates the proportion of variance in the criterion variable accounted for by combo of predictor variables * Ranges from 0 to 1, w/ higher scores meaning the IVs provide a good fit to the data

Answer 103

**When to use?** if you have a large number of potential predictors, but want to use a small subset of them for the final equation **Forward Step Wise MR:** start with one predictor and add others to the equation one at a time. Check predictive power after each one. MOST COMMON ONE **Backward Stepwise Regression:** start with all potential predictors, remove them on at a time and check for predictive power.

Answer 104

* This is used when there are multiple criterion and multiple predictor variables, and you want to understand their overall relationship * Looks @ two SETS of variables * Creates LINEAR COMBINATIONS of the variables that are maximally correlated w/ one another

Answer 105

* Creates discriminant functions (linear combos of the IV's) that best distinguish between groups * **Used when**: to classify cases into groups, or find which variables best differentiate between groups * **WIlks Lambda** test looks at how well a DFA separates the groups * **Eigenvalues**: amount of variance in DV explained by DF * **Canonical Correlation**: tells you ow well IV's explain group diffs **How is it different from multiple regression?** It predicts criterion GROUP rather than criterion SCORE E.g. high achievement or low achievement group, rather than specific scores

Answer 106

A characteristic in which the predictors involved in classifying people into criterion groups should have a different correlation with each criterion variable E.g. if you are trying to predict which major a uni student will excel in, the predictor variables should all be different to differentiate between possible groups (english, science, etc)

Answer 107

**Used for:** make predictions about which criterion group a person belongs to **How is it different from Discriminant Function Analysis?** * doesn't rely on the same assumptions * predictors can be nominal (categorical) or continuous **Use when:** * with dichotomous dependent variables (e.g. responder/non-responder to therapy) **Scores:** 0-1 0.80 = 80% chance of being a responder

Answer 108

1. **Normality** 2. **Homogeneity of Variance-Covariance Matrices** 3. **Independence**: observations independent of each other 4. **Linearity**: relationships between IV's linear

Answer 109

Cut offs are used for each predictor, and missing the cut off of even one predictor eliminates you E.g. job selection in which you need ALL eligibility criteria Compared to multiple regression, in which high scores on one predictor can compensate for lower scores on another (e.g. GRE scores)

Answer 110

If a relationship between two variables is obtained, but you suspect that the relationship may be due to another variable, you can 'partial out' its effect E.g. partially out hot weather in the correlation between ice cream and boat accidents

Answer 111

This is a spurious/extraneous variable that reduces the correlation rather than inflate it E.g. reading skill on a test impacting a job that doesn't require reading skill

Answer 112

**What is it?** A general term for techniques that are based on correlations between multiple variables **Assumptions:** linear relationship between variables **Used for:** testing causal models based on multiple variables

Answer 113

1. **Specify a causal model involving many variables**: IQ -> education -> empathy -> parenting -> childrens IQ 2. **Conduct Stat Analysis:** correlation between all pairs of variables 3. **Interpret Results of Analysis:** show if the data are consistent with the model

Answer 114

Verify causal models that propose one-way causal flows between variables Can be used only with observed variables (what you measure)

Answer 115

can be used with one-way and two-way causal relationships E.g. prediction that self esteem increases work success, which in turn leads to more self esteem Uses observed and latent (inferred) variables

Answer 116

**Used when:** * both variables are quantitative (interval/ratio) * interested in the trend of change rather than magnitude **Break points:** point where the scores for subjects change direction in a predictable way **What does it tell you?** what trends are significant

Answer 117

1. **Population**: whole set of something the research is interested in 2. **Sample Distribution**: set of scores obtained from a sample of a population 3. **Sampling Distribution**: multiple samples taken from population, and the means of those multiple samples are used to create a frequency distribution *samples must be same size *each population member must have same probability of being selected

Answer 118

When you pick a sample from a population, record the mean of that sample, and then return the sample to the population before you select your next sample The ones you just put back have the same probability of ending up in the next sample as do all the rest

Answer 119

1. As sample size increases, shape of the sample distribution approaches normalcy. True even if the population distribution of scores isn't normal 2. The mean of the sampling distribution is equal to the mean of the population

Answer 120

1. Sample distribution has less variability than population distribution 2. SD of sample distribution is equal to population SD divided by square root of the size of the samples from which means were obtained

Answer 121

When the rate of Type I error's is not increased by violations of the assumptions of parametric stat tests Central Limit Theorem is why parametric tests are robust with normality assumption, provided that the sample size is adequate enough to bring normality to the sample distribution Homogeneity of variance assumption: parametric tests are robust so long as equal no. of subjects in each experimental group

Answer 122

You don't need to have independence of observation to use this test (as opposed to t-tests) **Autocorrelation:** correlation between observations at given lags (e.g. between observations re: lag)

Answer 123

This is a formula used to get a special type of conditional probability E.g. probability that an 85yo has Alzheimer's given that they came up positive on a diagnostic test? -Basically, what is the probability that they have it and not a false positive?

Answer 124

Multiple studies analyzed at once, each study becomes a separate subject **Effect Size:** indicates magnitude of IV effect calculated for each DV, then summed and divided by # of effects It's the difference between the means of the control group and treatment group, divided by SD of control group