research design Flashcards
This flashcard deck was created using Flashcardlet's card creator
0
Q
treatment variable
A
one group receives treatment, another’s treatment is witheld. The treatment group is manipulated by the researcher. A treatment variable is measured in categories, treated and untreated.
1
Q
moderating variable
A
new variables constructed by the researcher to determine the joint impact of an independent variable on a dependent one. The moderating variable affects the relationship of the relationshipof the independent and dependent. acts upon another variable.
alters the relationship between independent and dependent variable
2
Q
control variable
A
a variable that the researcher does not want to measure directly. The control variable is neutralized by statistical procedure. Common control variables: gender, intelligence, socioeconomic status
3
Q
independent variable
A
Independent variables are located in the purpose statement , hypothesis, and research questions
Independent variables influence or affect a dependent variable ( outcome).
4
Q
dependent variable
A
a variable which is dependent on or influenced by an independent variable.
5
Q
mediating variable
A
aka intervening variable
can have an indirect effect on the dependent
the effect of x ( independent) on y (dependent) is mediated through m ( mediating)
6
Q
interrater reliability check
A
.
7
Q
researcher reliability
week 3 ppt
A
.
8
Q
metasynthesis
week 3
A
.
9
Q
meta analysis
week 2?
A
.
10
Q
unit of analysis
A
11
Q
constitutive definition
A
dictionary definition,
data quality, procedures, abstract
12
Q
operational definition
A
measures(data)
13
Q
Extraneous variable
A
In experiments, any aspect of the situation other than the treatment variable ta
hat can influence the dependent variable and that, if not controlled, canmake it impossible to determine whether the treatment is responsible for any observed effect on the dependent variable.
14
Q
confounding variable or factor
A
unintended differences in the way sample groups are treated. The resulting factors in the systematic differences are confounding factors e.g. differences in room temperature, time of day, room lighting.
15
Q
Random selection does not control for____________variables.
A
extraneous
16
Q
matching variables
A
matching variables by pairs to control extraneous variables
matching should be based on relevant variables, still considered random assignments
matching variables must be related to the dependent variable
creates a fair comparison and uses a control variable
match treated/untreated
increases internal validity
17
Q
Analysis of covariance is a method that can help control_____________ variables.
A
extraneous
18
Q
MANCOVA
A
more than 1 dependent variable
Multivariate Analysis of Variance
19
Q
ANOVA
A
Analysis of Variance
procedure for determining whether or not the difference between the mean scores of 2 or more groups on a dependent variable is statistically significant.
Can also be used with several levels of IV to determine whether each level/factor have a significant effect on the dependent variable.
20
Q
ANCOVA
A
Analysis of Covariance
Procedure used to determine whether the difference between the mean scores of 2 or more groups on one or more dependent variables is statistically significant, after controlling for initial differences between the groups on 1 or more extraneous variables. Similar to ANOVA in multi IV level/factors, after controlling for extraneous variable(s).
21
Q
Does increased sample size help control extraneous varianles?
A
NO
22
Q
Is restricting the sample a method that can help to control extraneous variables?
A
Yes, by limiting the variables.
23
Q
constitutive definition
A
using abstract words to describe abstract concepts
self concept
Examples:
parents participationin their kids school work
level of success in an academic area
24
operational definition
using measurable indicators to define a construct
Examples:
scores on parental involvement inventory, such as a scale
scale scores on FCAT
25
interval data
.
26
nominal data
sd not appropriate
27
ordinal data
strongly agree, disagree etc. ; can also be interval data
28
simple linear regression
1 independent, 1 dependent variable
an equation characterizing the straight line (linear) relation between the number of events and final exam scores can be determined
29
1 time per week
2 times per week
3 times per week on a survey would be what sort of data?
ordinal data
30
validity
content or judgement;
predictive or correlation;
concurrent or correlation
31
reliability
test, retest;
internal consistancy: Cronbach's alpha (or KR-20);
1.0 is perfect, .95 is very good, .01 not so much
32
Cronbach's alpha (function)
```
used to test reliability on questionaireand survey items
between 0-1.0
>.9 excellent
>.8 good
>.5 acceptable
>.3 questionable
>.2 unacceptable
SPSS : corrected Item Total
negative score is not good ( reliability);
used for reliability analysis;
spss code found in week 4 ppt;
lowest value will be the weakest item;
.509 is "fair"
many good items will help reliability,
```
33
Chronbach's alpha (description)
Corrected Item Total Correlation (SPSS)
Large numbers are positive and strong,
the weakest item has the smallest value.
34
factor analysis
internal structure of a test
| factor = a latent variable
35
common factor analysis
principal components analysis
36
correlation matrix vs. variance- covariancematrix
.
37
factor variables
latent, unseen variables ( as opposed to observed variables)
38
beta coefficient
pattern coefficient
39
correlation coefficient
structure coefficient
40
beta vs correlation coefficient?
The difference is in what is being controlled.
41
scree plot
correlation plot graph based on Eigenvalues greater than 1
| listed in descending magnitude
42
Eigenvalue
good value
(German)
used in qualitative analysis
43
exploratory factor analysis (efa)
uses an input a matrix of association. Matrix are Pearson moment Product and correlation
44
factor rotation
oblique (varimax)
| orthagonal (promax)
45
what is communality
```
R2= .01
R2 = .64
```
46
difference between pattern and structure coefficient?
structure only controls for --------- variables
47
test score reliability
refers to the consistency of test scores
48
reliability coefficient
values between 0 ( bad) and 1(perfect)
49
internal validity
internal validity refers to the extent to which we can accurately state that the independent variable produced the observed effect
50
external validity
relates to generalizing your findings
- to or across populations
- to or across tasks
- to or across environments
51
categorical variable
.
52
continuous variable
.
53
manipulated variable
variable can be changed or controlled by the researcher e.g. treatment/ no treatment
54
extraneous variable
The variable is related to the dependent variable, but not the focus of the study.
55
true experiment
must have at least 1 manipulated variable and random sampling
56
quasi-experiment
must have at least 1 manipulated independent variable, does not need to have random sampling
57
standard deviation of 0
all scores are the same
58
cognitive interview
.
59
probability sampling
quantitative
60
Convenience Sampling
.
61
Stratified Sampling
.
62
Cluster Sampling
involves random sampling of units like schools, cities
| probability sampling
63
Purposive Sampling
qualitative
64
phenomenological
.
65
unit of analysis
What am I studying?
Studies can have more than 1 unit of analysis
can have many levels
hierarchical: district>school>teacher>student
define inclusion criteria, exclusion criteria, recruitment procedures, sample size
66
Types of Probability Sampling:
Random Sampling
Stratifies Sampling
Cluster Sampling
Sampling Using Multiple Probability Techniques
67
types of Purposive Sampling
Sampling to achieve representativeness or comparability
Sampling special or unique cases
Sequential sampling
Sampling using multiple purposive techniques
68
Types of
| convenience sampling
captive sample
| volunteer sample
69
types of Mixed Methods Sampling
```
Basic Mixed Methods Sampling
Sequential Mixed Methods Sampling
Concurrent Mixed Methods Sampling
Multilevel Mixed Methods Sampling
Combination of Mixed Methods Sampling Strategies
```
70
gender: male/ female
grade: freshman, sophomore, junior, senior
gender and grade are independent variables
male/female or freshman/ sophomore.... are LEVELS or categories of one independent variable
2-way ANOVA will provide info on all levels
71
dependent t tests
or paired t test
repeated measures t test
one group and a pre-test, post-test
72
t test
the t test employs the statistic t to test a statistical hypothesis about the mean
(s) of a population
73
factorial ANOVA
.
74
Methods section of a proposal
must give specific methods for analyzing data
75
multiple regression
multiple continuous variables
76
case study
formal study to understand issues intrinsic to the person, group, event, etc.
77
ethnographic study
A study to describe and interpret the cultural behaviors, customs, and experiences. Process and product of describing cultural behavior.
78
phenomenological inquiry
study to describe the meaning of lived experiences for selected individuals or groups (e.g., bully victims, profoundly gifted adolescent males)
79
multilevel or heirarchical study
nested structure
80
high inference coding
more challenging re: reliability
more abstract, INFERS causality
more subjective
81
low inference coding
more black and white, higher reliability, not abstract, clear
82
APA tables
```
title left justified
table notes
review APA guidelines
all tables need to stand alone; acronyms need to be fully described APA= American Psychological Association
week 5 Thursday
```
83
centered value
age - the mean
| depression level - the mean
84
multicollinearity
the predictor variables are highly related, creating redundancy
85
causal statements
.
86
beta
slope
87
alpha
intercept
88
create a composite score for integrity scale
a composite score will be computed by taking an average of item responses. At least 3 items need to be answered. If not enough scores can be found, that participant will be coded as " missing data".wed week 7
89
F statistic
named for statistician R.A. Fisher, the statistic F is the ratio estimations of a population's variance, based on the information in two or more random samples. When used in ANOVA procedure, the obtained F value provides a test for the significance of the observed differences among the means of 2 or more random samples.
90
normal distribution curve
.
91
random sample
a set of items that have been drawn from the population in such a way that each member of the population has an equal chance of being selected. The population should be large enough that there is not a need to use replacement sample procedures.
92
F test
uses the statistic F to test various statistical hypotheses about the means of the distributions from which a sample or a setof samples have been drawn. The t test is a special form of the F test.
93
correlational design
aims to examine the relationship between 2 or more variables to see if they covary or correlate. CORRELATION DOES NOT INDICATE CAUSATION!
94
causal comparative design
.
95
quasi-experimental design
.
96
true experiment
.
97
meta-analysis
.
98
primary analysis
.
99
secondary analysis
.
100
correlational design
these studies examine the relationship between 2 or more variables
includes Pearson correlation r
simple linear regression
multiple linear regression
101
can you conclude that --------- produced the change?
no, because there are mant threats to the internal validity; there may be other events or situations that can explain the change.
102
common threats to internal validity: history
history: events that occured during the time between the pre and post- test
history is not the background of the participant
103
common threats to internal validity: maturation
mental, physical maturation can cause changes
104
common threats to internal validity: pre-test
it is possible to learn from the pre-test if using the same or similar post-test
105
common threats to internal validity: regression
extreme scores will regress to the mean. Changes upward may not be caused by increased learning, but from regression the mean
106
common threats to internal validity: measurement instrument
rules of measurement change, observer interpretation can be changed
107
common threats to internal validity
make sure to
108
common threats to internal validity: mortality or attrition
e.g., low achievers dropped out of study; overall post-test scores my be higher as a result
109
common threats to internal validity: solutions
use a control group and use random assignment
new program, old program
true experiment
compare treatment= control
use 1 way ANOVA ( 3 or more groups) or independent t test for 2 groups
110
3 way AVOVA
3 levels, one variable
111
advantages of pre-test
you can compare the groups at the start to be sure that the groups are similar at the beginning
you can pair and match by similarities (low readers, high) then randomly assign
propensity scores used for matching variables using logistic regression. Resulting scores will
you can use Statistical Control in ANCOVA
control ( minimize them) of extraneous variables help internal validity in a 1 group pre-and post-test
112
propensity scores
propensity scores used for matching variables using logistic regression
113
experimental design
researcher manipulates at least 1 independent variable
controls other variables
on completion of study, change can be attributed solely to the treatment
must include random samples
114
manipulation of the independent varianle
active independent variable
different treatment conditions
difference in time or degree
age, ethnicity, social status, and gender, ed. level, is never a manipulated variable
the researcher
will give or take away a treatment
variables are not able to control the variable
115
assigned or attribute independent variable
variable that cannot be manipulated by the researcher
116
variable
to be a variable, must have 2 attributes
| 14 girls, white socks and black socks
117
interval scale
equal intervals
lacks a true zero point
measures magnitude
temperature
118
nominal scale
numbers have no meaning
categories
coding, 1 and 2
119
ways to improve internal validity
```
matching
include additional independent intothe design
use statistical control like ANCOVA
repeated measures
random assignment
homogeneous selection
```
120
random sample
the group must be formed first, then randomly assign participants.
choosing groups as people enter a room is not random.
121
non-manipulated variable
attributed variable
| researcher does not have control over
122
true experiment that uses statistical control
.
123
ordinal data
anything that is rank ordered
| Likert scale
124
ratio scale
| ratio data
```
absolute zero is possible (lack or absence of)
has a starting point of zero
equal intervals
measures magnitude
height, weight, age in months or years
percentages
yardstick
```
125
statistical control
.
126
repeated measures design
each participant will receive multiple treatments
individual times will be compared amongst their own scores
can be counterbalanced
127
validity of design
internal
external- valid for another population, generalization see GGB
ecological- need complete description of environmental conditions
e.g., noise, light, warmth, etc.
128
non-probability sample
.
129
explanatory vs predictive
.
130
how to determine statistical power
calculate effect size, use a power table that matches sample size, stated alpha= effect size
131
Cohen's D
.
132
use Chronbach's alpha for reliability
.
133
correlation vs causal comparative design
correlational variables are continuous
causal- IV is categorical <. 20 - > 10
the IV are not in the researcher's control, therefore, experimental and even quasi- experimental designs will have stronger internal validity
134
heuristic
"to find or discover"
experienced-based techniques for solving a problem e.g. trial and error
speculative formulation which serves a guide in an investigation
135
type 1 error
the rejection of the null hypothesis when it is true
136
type II error
the acceptance of the null hypothesis when it is false
137
triangulation
the use of multiple data-collection methods, data sources, analysts, or theories as evidence for the validity of qualitative research findings
138
internal validity
.
139
external validity
.
140
Threats to internal validity
| History
events occuring during the time between the pre-test and post-test
141
Threats to internal validity
| Maturation
the subjects/samples developed or matured; effects in improvement may be attributed to natural conditions of maturation and cognitive development.
142
Threats to internal validity
| pre-testing
when tested twice, learning may occur during the first test
143
Threats to internal validity
| statistical regression
extreme scores tend to regress to the mean
144
Threats to internal validity
| measurement instrument
the researcher used a different test form for the pre and post test.
improvement cannot be solely attributed to the intervention program.
145
Threats to internal validity
| attrition or mortality
students drop out of program or die
146
Threats to internal validity
| solutions
use a control group
| use random assignment
147
one-way ANOVA
1 categorical independent variable with levels
148
advantages of pre-testing
good for use with unequal group sizes
| compares groups at the beginning of study, then growth means between groups can be fairly compared
149
Advantages if pre-tests
matching in pre-tests
enhanced internal validity
similar participants can be matched between groups in pre-tests for better internal validity
150
Advantages if pre-tests
if you have a pretest, you can use statistical control (ANOVA)
control of extraneous variables increases internal validity
if people drop out, you will learn something about them
151
disadvantages of pre-testing
students learn from the test, descreasing internal validty
| it takes time to test, reducing instructional time
152
correlational data
often complex
NOT CAUSAL
no manipulated variable
should not use the word "effect", rather, use "relates to"
153
ratio data
| ratio scale
0 means 0 events
interval and ratio data are often used concurrently
true zero value
154
correlation high strong or low weak when MEAN is the same, but SD is larger
the correlation will be stronger if there is a wider spread
155
final exam scores ranged from 54-98
| what type of measurement scale?
interval or ordinal data,
156
4 conditions needed to infer causality
1. Time order
2. covariation (variables must correlate)
3. conceptual/logical explanation
4. other explanations HAVE to be ruled out
157
r
correlation
158
r and Standard Error of Estimate
.
159
multicollinearity
.
160
intercept
not a variable, just the point that x and y intercept
161
Regression Model
see Ellum July 11
162
amount of tv viewing
| what type of measurement scale?
ratio: a true zero is possible. No tv hours
163
reading comprehension test
ordinal or interval
| not ratio, zero knowledge is not likely
164
probability sample
random sample, higher population validity
165
non-probability
.
166
pattern and structure coefficients
concepts related to factor analysis
167
factor analysis
you will have columns
themes
rows and columns for factors and items in a survey or questionaire
168
pattern coefficient
loadings
where does an item load on an item, controlling for the other factors
a standardized regression coefficients
169
structure coefficient in a factor analysis
correlation between factors and items
170
concurrent validity
2 measures of the same thing or at the same time
171
predictive validity
x predicts y
172
factors are observed or latent variables?
latent
173
orthogonal rotation varimax
| uncorrelated factors
.
174
Cronbach's alpha coefficient
A measure of the internal consistancy of a test, based on the extent to which test takers who answer a test item one way respond to other items the same way.
175
correlation coefficient
A mathmatical expression of the direction and magnitude of the relationship between 2 measured variables.
176
Deception
The act of creating a false impression in the minds of research participants through such procedures as withholding information, providing false information, creating false intimacy, or using accomplices.
177
correlation matrix
an arrangement of correlation coefficients in rows and columns that makes it easy to see how each of a set of measured variables correlates with all the other variables.
178
grounded theory
An approach to theory development that involves deriving constructs and laws directly from the immediate data that the researcher has collected rather than drawing on an existing theory.
179
R
multiple correlation coefficient
180
r or Pearson r
product-moment correlation coefficient
Pearson product
a mathematical expression of the direction and magnitude of the relationship between 2 measures that yield continuous scores.
181
R2
coefficient of determination
182
loading
In factor analysis, the degree and direction of the correlation of the correlation between a measured variable and a particular factor.
183
collinearity
The degree of correlation between any two variables that are to be used at predictors in a multiple regression analysis.
184
probability sampling
a procedure for drawing a sample from a population such that each individual has a known chance of being ses
lected.
185
probability (p)
the chance that a statistical result was obtained by pure chance
186
exploratory data analysis
Exploratory data analysis is a method of discovering patterns in
a set of scores.
187
Factor analysis
a statistical procedure for reducing a set of measured variables to a smaller number of factors or latent variables.
188
exogenous variable
in path analysis, a variable for which there is no other variable in the model that is hypothesized to influence it.
189
ecological validity
the extent to which the results of an experiment can be generalized from conditions in the research setting to particularly naturally occurring conditions.
190
nominal data
| nominal scale
a measure in which numbers represent categories that have no rank order or quantitative value, coding of categories such as gender, race, grade level
191
path analysis
statistical method for testing the validity of a theory about causal links between three or more measured variables.
192
path coefficient
in a path analysis, a standardized regression coefficient that expresses the degree of direct effect of one measured variable on another measured variable.
193
phenomenology
the study of the world as it appears to individuals when they place themselves in a state of consciousness that strives to be free of everyday biases and beliefs.
194
retrocausality
| retro-causation
allowing an effect to occur before its cause
195
path analysis
used when the theoretical causal paths are being investigative
196
non-directional hypothesis
"I predict that tv and test performance will be significantly related".
197
directional hypothesis
I predict that as tv watching increases, test performance will decrease
predicts a direction or magnitude
198
Likert scale, continuous or categorical?
catergorical, over 9 (some may say 5) variables could be considered continuous
199
scale of measurement for an exam
interval or ordinal
| not ratio...or complete lack of knowledge
200
less variability in data, strong or weak correlation?
| high or low reliability?
weak correlation with a cluster of data ( not spread out)
| low reliability with less score spread
201
true experiment
manipulated variable
| random assignment
202
quasi experiment
manipulated independent variable
| not random assignment
203
correlational design
```
independent variable not manipulated
not randomly assigned
does not indicate causality
often misinterpreted
regression analyses
continuous data
```
204
causal comparative
independent variable not manipulated
not random
categorical data
205
test-retest
| validity or reliability?
.
206
in a perfectly reliable test, what would the standard error of measurement be?
0
207
high reliability coefficient in a random sample or homogeneous sample?
random selection will increase spread and increase test reliability
208
concurrent validity
correlating scores on a self report and observer's ratings on a similar test
209
if everything else was the same, which would have the highest reliability?
a test with 5 questions, or a test with 20 questions?
the test with 20 items.
| more items = more score spread= higher reliability
210
which is the strongest correlation?
.57
.90
-1.0
-1.0
correlations can be strong positive, 1.0 or
strong negative, -1.0
211
variance
the average squared deviation of each number from its mean
212
threats to external validity
pretest sensitization
multiple treatment interference
treatment setting interaction
213
threats to internal validity
```
history
maturation
testing
regression
selection
mortality
```
214
concurrent evidence adds to validity or reliability?
validity
215
```
in perfect reliability, the standard error of measurement would be
1
100
-1
0
```
0
216
effect size
difference between the means of groups
range from 0 ( no difference)
to 3.0
2.0 is a large effect
217
quota sample
non-probability sample; similar to stratified, but not randomly assigned
218
stratified sample
probability sample
| random assignment of subgroups
219
systematic sampling
probability sampling
| randomly picking a starting point and taking every kth case
220
survey
study of a sample or part of the population
221
census
study of an entire population
222
random selection
external validity
223
random assignment
internal validity
