Terminology and Statistics Flashcards Preview

Research > Terminology and Statistics > Flashcards

Flashcards in Terminology and Statistics Deck (61):

Between groups

looking at differences BETWEEN different groups

ex. Group 1 vs. Group 2


Within groups

-repeated measures
looking at differences WITHIN (same person over time)

ex. one participant: T1 vs T2 vs T3


Analysis of Variance (ANOVA)

- One- way = 1 IV
- Two-way = 2 IV
- Factorial = 2+ IV


F statistic

-Ratio of the variation explained by the model and the variation explained by unsystematic factors
-If an f statistic is 1 or greater, chances are it will be significant


The OVAs

ANOVA- analysis of variance
ANCOVA- analysis of covariance (1+ covariate)
MANOVA- multivariate analysis of variance ( 2+ DVs)
MANCOVA- multivariate analysis of variance (2+ DVs, 1+ covariate)
-Mixed- model design ANOVA


Mixed model design ANOVA

-gets its name because there are two types of variables involved, that is at least one between-subjects variable and at least one within-subjects variable.
-The mixed-design ANOVA model (also known as Split-plot ANOVA (SPANOVA)) tests for mean differences between two or more independent groups whilst subjecting participants to repeated measures. Thus, there is at least one between-subjects variable and at least one within-subjects variable.


Example of a mixed model design ANOVA

-For example, are there any differences amongst the heights of males and females at age 10 and age 20 years?
-Gender (male or female) is the between-subjects variable
-Age (10 or 20 years) is the within-subjects variable
-Of interest are the main effects for Gender and Age, and the Gender-Age interaction effect.
-This could be described as a 2 x (2) mixed-design ANOVA


Main effect

with example

-the overall effect of one independent variable.
*EXAMPLE: In an experiment in which both the type of psychotherapy (cognitive vs. behavioral) and the duration of psychotherapy (short vs. long) are independent variables, there is one main effect of type and another main effect of duration. The main effect of type is the difference between the average score for the cognitive group and the average score for the behavioral group … ignoring duration. That is, short-duration subjects and long-duration subjects are combined together in computing these averages. The main effect of duration is the difference between the average score for the short-duration group and the average score for the long-duration group … this time ignoring type. Cognitive-therapy subjects and behavioral-therapy subjects are combined together in computing these averages.


Interaction effects

with example

-In order to find an interaction, you must have a factorial design, in which the two (or more) independent variables are "crossed.” a special kind of effect that can be observed in factorial experiments. You have an interaction whenever the effect of one independent variable depends on the level of the other. This is actually a fairly easy idea.

Here are some examples. The combined effect of two or more variables, as demonstrated in a factorial design; interactions signify that the effect of one variable (e.g., sex of the subject) depends of the level of another variable (e.g., age)
-If cognitive psychotherapy is better than behavioral psychotherapy when the therapy is short but not when the therapy is long, then there is an interaction between type and duration of therapy.
-If the negative effect of noise level on concentration is greater for introverts than for extroverts, then there is an interaction between these two independent variables.
-If the boost in intelligence judgments due to smiling is greater for male stimulus persons than for female stimulus persons, then there is an interaction between smiling and sex.
-If drawing a smiley face on checks increases tips for female servers but not for male servers, then there is an interaction between drawing smiley faces (or not) and sex of the server.


crossover interaction

-the effect of one independent variable is not only different across levels of the second independent variable, it actually reverses. Imagine, for example, that introverts’ concentration levels started high and then dropped as the noise level increased, but extroverts’ concentration level started low and then increased as the noise level increased. This would be a crossover interaction.


Buzz words:

predictors =
relationship =
adjusting =
post-hoc =

three groups


Discriminant Function Analysis (DFA)

-quantitative variable to predict group membership (Categorical)
-reverse MANOVA


homoscedasticity (homogeneity of variance)

-Condition in which all the variable in a sequence have the same finite, or limited, variance
-When homogeneity of variance is determined to hold true for a statistical model, a simpler statistical or computational approach to analyzing data may be used due to a low level of uncertainty in the data


Two types of covariates:




a variable that INFLUENCES the strength of a relationship between two other variables

example: in arguments/debates, the two parties agree from the beginning to have a conversation. The moderator is simply there to help manage the strength of the two parties' arguments. i.e., this variable has an effect for sure, but the relationship will be there one way or the other



a variable that EXPLAINS the relationships between two other variables

- in disputes, mediators are usually called in when two parties are unable/willing to communicate. It all hinges on the mediator to help solve the problem. i.e., if the mediator is gone, the relationship is nonexistent!


Example of mediator vs moderator

-What's the relationship between social class (SES) and frequency of breast self-exams (BSE)? Age might be a moderator variable, in that the relationship between SES and BSE could be stronger for older women and less strong or nonexistent for younger women. Education might be a mediator variable in that it explains why there is relation between SES and BSE. When you remove the effect of education, the relationship between SES and BSE disappears.


confounding variable

a variable that is influencing both the DV and IV and is not accounted for. Failure to account for confounding variables can lead to spurious relationship
Example: ice cream sales impact drowning deaths. This is a spurious relationship because it fails to account for season



a variable that has a relation with one or both of the IV and DVs, but does not appreciably change the relation between an IV and DV when included in a statistical analysis. Covariates are generally not of theoretical interest, but are often included in a model to explain additional variability in a DV



used in prediction and forecasting

Types: linear, bivariate, logistic, quadratic, exponential, logarithmic, power, etc.)


bivariate regression

degree of relationship between two quantitative variables


logistic regression

degree of relationship between categorical variables


linear regression

when both the predictor and response variables are continuous and linearly related, so the response will increase or decrease at a constant ratio to the predictor. There can be more than one predictor


example of linear regression

As the number of farms has decreased in the U.S., the average size of the remaining farms has grown larger. Based on data (year and the corresponding average acreage/farm) predict the average acreage in 2000 and 2010 and determine which function gives the most realistic predictions


logistic regression

- the predictor is continuous, but the response is categorical or dichotomous. The logistic regression provides you with the probability of an event occurring. For every one unit of increase in the predictor, the probability of an event occurring increases or decreases.
-The information of interest is the probability of an individual falling into one of the two populations given the scores of the IVs
-Originally used in medical research where the DV was the presence or absence of a disease and the IV's were risk factors


example of logistic regression

Suppose that we are interested in the factors that influence whether a political candidate wins an election. The outcome (response) variable is binary (0/1): win or lose. The predictor variables of interest are the amount of money spent on the campaign, the amount of time spend campaigning negatively, and whether or not the candidate is an incumbent.


Statistical conclusion validity

the extent to which a relation between independent and dependent variables can be shown, based on quantitative and statistical considerations of the investigation. Although Cook and Campbell seemed to base this on the preponderance on inferential statistic-based research in psychology, Kazdin notes that not all psychological research is based on inferential statistics (e.g., qualitative research and single-case experimental designs). He therefore recommends the term, data evaluation validity


Threats to statistical conclusion validity

Mondays: Law and order: SVU

-Multiple comparisons and errors rates
-low statistical power
-subject heterogeneity
-variability in the procedures
-unreliability of the measures


Multiple comparisons and errors rates

Threat to statistical conclusion validity

-When there are multiple comparisons, alpha increases beyond point .05
E.g., if you gave different measures to clients, clinicians, and client family members, on a variety of areas, the likelihood that there will be differences across the ratings increase by chance alone


low statistical power

Threat to statistical conclusion validity

most common threat
the study is not designed to detect a difference when there very well may be one


subject heterogeneity

Threat to statistical conclusion validity

the greater the diversity of the sample, the more difficulty it will be to detect a difference between groups


variability in the procedures

Threat to statistical conclusion validity

While not all variability can be controlled for, failing to minimize variability will impact effect size, biasing results
Pg. 72, “In terms of our formula for effect size, the differences between groups will be divided by a measure of variability; this measure will be larger when there is more, rather than less, uncontrolled variation. The larger the variability, the lower the effect size evident for a given difference between groups.


unreliability of the measures

Threat to statistical conclusion validity

self explanatory


Construct validity

(Kazdin, 2003)

In the context of experimental design, this refers to a type of experimental validity that pertains to the interpretation or basis of the effect that was demonstrated in an experiment. (in the context of a psychological assessment the term refers to extent to which a measure has been shown to assess the construct [e.g., intelligence] of interest)


Threats to construct validity


-Cues of experimental situation
-Attention and contact with clients
-Single operations and narrow stimulus sampling
-Experimenter expectancies


Cues of the experimental situation

Threats to construct validity

-Those seemingly ancillary factors associated with the intervention that may contribute to the results
-These cues have been referred to as demand characteristics (Orne, 1962)
E.g., information conveyed to prospective participants prior to their arrival (rumors about experiment, information provided during recruitment), instructions, procedures, and any other features of the experiment


Attention and contact with clients

Threats to construct validity

Attention to the clients, rather than the intervention itself, might be responsible for change
E.g., In placebo trials, for instance, the intervention of a drug may clearly work (internal validity); but that does not mean that the best conclusion is that it was the drug itself that yielded the results, rather than other aspects of the intervention.


Single operations and narrow stimulus sampling

Threats to construct validity

An experimental manipulation or intervention that includes features that the investigator considers to be irrelevant to the study, but these features later introduce ambiguity in interpreting the findings.
E.g., Psychodynamic therapy as measured by veteran analysts vs. CBT administered by social workers in their first year. Were the treatments being studied, or was this a study measuring the skills of the first group vs. the skills of the second?


Experimenter expectancies

Threats to construct validity

-The expectancies, beliefs, and desires of the experimenter (the one administering treatment) for the results influences how participants perform


Internal validity

(Kazdin, 2003)

– the extent to which the experimental manipulation or intervention, rather than extraneous influences, can account for the results, changes, or group differences.
-the "approximate validity with which we infer that a relationship between two variables is causal" (Cook and Campbell, 1979. P.37).
• A good synonym for the term internal validity is causal validity because that is what internal validity is all about.
• If you can show that you have high internal validity (i.e., high causal validity) then you can conclude that you have strong evidence of causality; however, if you have low internal validity then you must conclude that you have little or no evidence of causality


Threats to internal validity


-Diffusion or imitation of treatment
-statistical regression
-combination of selection and other threats
-selection bias-
-special treatment or reactions of controls


Diffusion or imitation of treatment

Threats to internal validity

experimental conditions are unintentionally similar to one another (e.g., behavioral therapy vs. eclectic, that unintentionally includes behavioral methods)



Threats to internal validity

the measuring instrument or measurement procedure can change over time (e.g., WAIS-R vs. WAIS-IV)


Statistical regression

Threats to internal validity

statistically, individuals initially testing with extreme scores tend to revert towards the mean during the second testing



Threats to internal validity

exposure to the test previously (as in pre-test/post-test) might account for a change in performance, not the experimental manipulation


Combination of selection and other threats

Threats to internal validity

when there are different internal validity threats for different experimental groups (e.g., group 1 = experienced the world trade center attacks, group 2 = did not experience this historical event)



Threats to internal validity

events both internal and external to the experiment not accounted for by the IV (and otherwise not controlled for); ex. what's going on in the news, weather, that may account for results



Threats to internal validity

loss of subjects over time (drop-out)


Selection bias

Threats to internal validity

systematic differences between experimental groups prior to the experiment



Threats to internal validity

changes over time (e.g. participants getting older, wiser, bored, etc.)


Special Treatment of reactions of controls

Threats to internal validity

 The control group is unintentionally given an incentive or condition that was not controlled for (e.g., higher compensation), that could be affecting results


External validity

the extent to which the results can be generalized or extended to persons, settings, times, measures, and characteristic other than those in this particular experimental arrangement.


Threats to external validity


-Multiple treatment interface
-reactivity of experimental arrangements
-sample characteristics
-test sensitization
-reactivity of assessment
-novelty effects
-timing of measurement
-stimulus characteristics


Multiple- treatment interface

threats to external validity

to what extent does the interaction of receiving multiple experimental conditions (i.e. such as in a within subjects design) affect the results?


Reactivity of experimental arrangements

Threats to external validity

-How does the fact that one knows they are in an experiment affect the results
Pg 95 of Kazdin
Good – find out hypothesis (you think), and try to collaborate with it
Negativistic – find out hypothesis (you thin),and try to contradict it
Faithful – comply with all instructions
Apprehensive – try to portray yourself favorably


sample characteristics

threats to external validity

do the characteristics of the sample limit generalizability?


Test sensitization

threats to external validity

Giving a pretest might sensitize a person to the construct you are measuring (e.g., giving a Beck Depression Inventory before the participant watches a sad film might alert them to depressing stimuli, confounding results)


Reactivity of assessment

threats to external validity

this is participant knowledge that they are being assessed in the experiment


Novelty effects

threats to external validity

interaction with unfamiliar stimuli might be more responsible for results (e.g. imagine an experiment where someone had to remember a neon-pink dinosaur versus a picture of a chair)


Timing of measurement

threats to external validity

Does the timing for assessing experimental group bias results (e.g. evaluating therapy effectiveness a year from termination as opposed to 6 months)


Stimulus characteristics

Threats to external validity

to what extent is the setting affecting the results (e.g. for treatment studies, controlled clinical trials do mimic real-world clinical practice settings)