Research and Program Evaluation Flashcards
Research question
- relational, descriptive, causal
Quasi-experiment
- the researcher uses preexisting groups, hence the independent variable (IV) cannot be altered (example gender, age, etc). in this type of experiment, one cannot say with any degree of statistical confidence that the IV caused the dependent variable (DV)
Internal Validity
- whether the DVs were truly influenced by the experimental IVs
Threats to interval validity
- maturation of subjects, mortality (subject withdrawing), instruments used, statistical regression
External Validity
- whether the results can be generalized to a large population. e.g if the results of a study only apply to the population in the study then the external validity is low.
Chi-square
- non-parametric statistical measure that tests whether a distribution differs significantly from an expected theoretical distribution
- it’s used to determine whether an obtained distribution differs significantly from an expected distribution
Non-parametric
- only able to make a few assumptions
Factor analysis
- data reduction
Parsimony
- interpreting the results in the simplest way
Occam’s Razor
- suggests interpreting the results in the simplest manner
Bubbles
- flaws in research
Journal of Counseling Psych
- publishes more counseling research articles than any other periodical in the field
Confounding
- Occurs when an undesirable variable are not kept out of the experiment, flaw the experiment, all correlational research is said to be confounded
Basic Research
- conducted to advance our understanding of theory
Applied Research
- conducted to advance our knowledge of how theories, skills, and techniques can be used in terms of practical application.
IV
- variable research manipulates, controls
DV
- expresses the outcome of the data
Control group
- does not receive the IV (treatment)
Experimental group
- receives the IV
Ethics:
- subjects are informed of any risks
- negative after effects are removed
- allowed to withdraw at any time
- confidentiality of subjects are protected
- results reported in an accurate format
- Only use techniques you are trained in
30 people
- to conduct a true experiment
100 people
- for a survey
Organismic variable
- variable researchers cannot control yet exist. E.g: height, weight, gender, natural variable
R.A Fisher
- Hypothesis testing
- determining if the null hypothesis is to be accepted or rejected
Hypothesis
- educated guess
Research hypothesis
- testable expected relationship between two or more variables
Null hypothesis
- suggest that there will not be a significant difference between the experimental group and the control group
Meta-analysis
- study that analyzes the findings of numerous studies
Alternative hypothesis
- asserts that IV causes a change
Inferential statistical
- provide information about the population
Descriptive Analysis
- merely describe the data
T-test
- compares 2 groups
- used to ascertain whether 2 sample means are significantly different
two-tailed T-test
- non-directional experimental hypothesis
One-tailed T-test
- directional experimental hypothesis, the hypothesis specifies that one average mean is larger than the other
Between-subjects design
- when a research study uses different subjects for each condition, each subject receives only 1 values of the IV, exploring the effects of treatment between two groups.
Within-subject design
- same subjects are employed, 2 or more values/levels of the IV are administered to each subject, assess changes that occur as they experience the intervention
Parameter
- summarizes a characteristic of a population
Split-Plot design
- assess a general intervention on whole plot and other treatments to subplots within the whole
Ethnographic research
- involves information collected via interviews, observations, and the inspection of documents
P level
- level of significance/ level of confidence
- .05 on 5% chance the different between control and experimental group is due to chance factors
- .01
- .001 rules out chance levels - high confidence of less error
Type I Error (alpha)
- researchers rejects the null when it is true
- raising sample sizes lowers type 1 and 2 errors
Type II Error (beta)
- researchers accept the null when it is false
- raising sample sizes lowers type 1 and 2 errors
Anova
- more than 2 groups
- a one-way analysis of variance is used for testing one independent variable
Ancova
- analysis of covariance which tests 2 or more groups for extreme variable
Manova
- study has more than two DV
2 way Anova
- requires 2 IVs
Mancova
- involves multiple dependent variables
Correlation coefficient
- association between two variables, how a change in one is to the change in another correlation does not imply causation
Positive correlation
- both variable change in the same direction
Negative correlation
- variable are inversely associated
Biserial correlation
- 1 variable is continuous and the other is dichotomous
Hawthorne Effect
- reacting to the presence of a research
- affects participants responses
- if the subject knows they are given a particular attention they respond to that attention
Rosenthal Effect
- experimenter’s belief about the individual may cause the individual to be treated in a special way so that the individual begins to fulfill the experimenter’s expectations
Observer bias
- when the researcher has perceptions regarding the research that are inaccurate
Hallo Effect
- occurs when a trait which is not being evaluated influence a researcher’s rating on another trait
Trend Analysis
- a statistical procedure performed at different times to see if a trend is evident
Analysis of Covariances
- controls for sample differences which exist, helps, to remove confounding variable, it statistically eliminates
Statistical Regression
- threats to internal validity, predicts that very high and very low scored will move toward the mean it a test is administered again
Cross-sectional
- clients are assessed at one point in time
Longitudinal study
- the same people are studied over a period of time
Summative Evaluation
- is used to assess a final product and attempts to ascertain how well the goal has been met
Formative
- is going and conducted while the program is underway (progress evaluation)
Counterbalancing
- switching the order in which stimuli are presented to a subjected
Cluster sampling
- used when a list of the entire population cannot be found and solved this problem by using as existing sample or cluster of the population as a sample
Horizontal sampling
- occurs when a researcher subjects from a single socioeconomic group
Vertical sampling
- occurs when people from 2 or more socioeconomic classes are utilized
Systematic sampling
- take every nth person
Parametric test
- the assumption is that the scores are normally distributed
Non-parametric sampling
- the curve is not a normal distribution
independent groups
- unmatched or uncorrelated groups
organismic variable
cannot be manipulated like hight and weight
Null hypothesis
- asserts that the samples will not change
- IV does not affect the DV
test of significance
- determines whether a difference in the group scores is significant
- compare control group to an experimental group
social sciences
- accept probability level of .05 or less
Alpha
- Error I
Alpha
- Error I
- reject the null hypothesis when true
Beta
- Error II
- accept the null when false
.01 to .001
- lowest type I error, however, increases type error II beta
Nominal
- qualitative
- the most elementary
- classifies names, labels, or identifies by group
- has no true zero
- does not indicate order
Positive correlation
- both values go up
negative correlation
- one value goes up and the other down
behaviorist often utilize N =1
- take a baseline measure
Single blind study
- the subject does not know whether they are part of the control of the experimental group
Double-blind study
- when the subject is unaware of the status
- neither the subject not the research know who belongs to what group
Pearson r
- Interval and ratio
Spearman rho
- ordinal data
68-95-97.5
- SD: 1 -2-3
- ALL SCORES will fall between 3 SD s pf the mean
modal score
- the highest point of the curve
mode
- the point of maximum concentration
- highest point of distribution
range
- it’s the distance btw the largest and smallest number
mean
- average
median
- it’s the best for skewed distributions and extreme scores
Raw score
- means nothing
Transformed score or standard score
- such as percentile ranks, t-scores, z-scores, stanines, or standard deviation over a raw score is that standard score allows you to analyze the data in relation to the properties of the normal bell-shaped curve
X axis
- it’s also known as the abscissa
- plot IV scores
Y axis
- it’s also known as the ordinate
- plot the DV
Z Score
- the same as a standard deviation
- also called a standard score
T score
- mean = 50
- SD = 10
NOIR
- NOMINAL
- ORDINAL
- INTERVAL
- RATIO
Ordinal
- provides relative placement or standing
= order
Interval
- used for tests in schools
- no absolute zero
Ratio
- has a true zero point
- most psychological attributes cannot be measured on a ratio scale
- highest level of measurement
- time, height, weight, temperature volume, and distance meet the reqs for this measurement
Halo effect
- occurs when a trait that’s not being evaluated influences a researcher’s rating on another trait (e.g attractiveness or how much a person is liked)
Analysis of covariance techniques
- it helps to eliminate difference between groups
cohort study
- study with subjects that share same characteristics
Standardized tests
- always have formal procedures for test administration and scoring
- testing format, materials, and scoring process are consistent
cross-sectional method
- done at a single point in time and preferably in time consumption
Ipsative vs Normative
- ipsative implies a within-person analysis rather than a normative analysis between individuals
Duncan’s multiple range, Tukey’s, or Scheffe’s test
- to test significant differences between group means
Casual comparative research
- groups were not randomly assigned and the researcher did not control the IV
systematic sampling
- choosing every TH person
Operational definition
- outlines a procedure
- operationally define procedures so that other researchers can attempt to replicate
parametric tests
- normally distributed
nonparametric test
- not a normally distributed
Matched design
- subjects are matched in regard to any variable that could be correlated with the DV
U Mann-Whitney
- uncorrelated/unmatched
- independent groups
- two groups
Wilcoxon
- it’s an alternative to the t test when parametric precepts cannot be accepted
- can be used for two groups
deductive
- reduces the general to specific
inductive
- goes from specific to a generalization
Standard Error of Measurement (SEM)
- tells the counselor what would most likely occur if the same individual took the same test again
Likert scale
- includes choices and categories
