Module 4- Measurement Flashcards
1
Q
How do we test Hypotheses?
A
- Empirically measure the variables
- quantify the variables; put numbers to them so we can measure it
2
Q
Variable
A
- Any characteristic that can take on more than one value
- research is looking at the relationship between variables
3
Q
Measurement
A
- quantification of the amount of some variable that is present
- assign numbers to indicate the amount of variable that is present
- can be arbitrary numbers
4
Q
4 levels of measurement
A
- Nominal
- Ordinal
- Interval
- Ratio
5
Q
Nominal Level of Measurement
A
- Numbers assigned are arbitrary and do not represent any underlying quantitative aspect of the variable
- mathematical property= property identity
ex. groupings of gender, colour, if they did smthg or not
6
Q
Ordinal Level of Measurement
A
- ranking of the data but with no actual amount
- mathematical property= magnitude
- no assumption of equal intervals bw variables
ex. class scores ranked from 1-100, know who has more of the variable but don’t know the actual amount/ score
7
Q
Interval Level of Measurement
A
- shows the actual amount of variable present
- assumes equal intervals bw scores/ variables
- mathematical property= magnitude and equal intervals
- DO NOT HAVE A TRUE ZERO POINT
- ex. cannot have a zero IQ or heartbeat
8
Q
Ratio Level of Measurement
A
- mathematical properties= magnitude and equal intervals
- can have a true zero= absolute absence of the variable
ex. quiz would be a ratio measurement bc can get a zero and for it to be meaningful
9
Q
level of measurement we choose for our operational definition will determine…
A
- how precise our measurement is
- what statistical analysis we conduct
10
Q
Most precise levels of measurement are
A
Interval and Ratio
- allow for better statistical analysis
- most informative and precise measurement of the variable
11
Q
want to operational define our variables at which level?
A
- highest level as possible bc we want the most precise measurement
(Interval or Ratio Scales) - we can take interval and ratio data and collapse it down to lower levels BUT we cannot take nominal data (low level ) and expand it up
12
Q
2 dimensions of quality to asses if we are measuring our variables accurately/ how to evaluate measures
A
- Reliability
- Validity
13
Q
Reliability
A
- concerned with measurement errors
- concerned with reproducibility
- give consistent and reproducible results
14
Q
Validity
A
- concerned with to what extent do the scores represent the variable we intend to measure
- observed score reflect the intended construct
- are you measuring what you intended to measure
15
Q
connection bw reliability and validity
A
- measure can be reliable and not valid
- reliability is a pre-req for validity BUT does not guarantee validity
16
Q
Psychometrics
A
- focuses on judging and improving the reliability and validity of the psychological measures
17
Q
no measure is perfect
A
- yes
- measure contains elements of the construct and error
18
Q
True Score (T)
A
- error free score an individual would receive on a test if their was no measurement error
- individual score on a measure if no error
19
Q
Data point or observed score can be represented by the following formula
A
X= T+ E
- X; Observed Score
- T; True Score
- E; Error
conceptual formal; don’t plug in numbers
- shows that observed score is comprised of true score of the construct and error
20
Q
Random Error (ER)
A
- unpredictable and unsystematic
- does not impact all data points in the same way
- cancels out over a larger number of observations
- causes unreliability in measurements; unable to reproduce the observed score
21
Q
Systematic Error
A
- non random and predictable
- does not cancel out over a large number of observations
- does impact the scores of the entire group
- contributes to invalidity bc no longer measuring the construct of interest
- skews data in a predictable way
22
Q
2 types of systematic error
A
- Bias
- Error associated with measuring the wrong construct
23
Q
Bias (EB)
A
- measurements can be biases bc of situations or devices (equipment, judges, raters…)
-ex. difference is test difficulty and difference in testing conditions. Bias bc the data will be skewed in a predictable way
24
Q
Error
A
- difference bw the true score (actual amount) and the observed score (data)
25
How to deal with bias?
- calibrate devices and measure in the same way
- operational definitions are precise
- testing procedures are standardized
26
Error associated with measuring the wrong construct (EW)
- Measuring a theoretical construct other than the one we are interested in
- ex. when taking an exam, we might be measuring exam anxiety supposed to your understanding of the knowledge
27
conceptual formula including Ew and Er
X=T+Er+Ew
- note; assumed the researcher has taken care of bias so it is not included in the conceptual formula
28
Unreliability
- random error contributes to unreliability
- X=Er; measurement made of only random error and not measuring the construct
- random error is like meaningless noise than can't be reproduced (like static)
- X=Er, not tapping into the true or wrong construct; measuring only random error
29
why can't you reproduce random error?
- bc it is meaningless
30
Reliability conceptual formula
- X= T+ Ew
- no random error in reliable measures
- have Ew bc we could be measuring the construct of interest or another construct or a combo of both
- can reproduce the measure bc it is reliable
- ex. tune into a radio station and hear music. even though we are not sure if it is the station we want, we can reproduce the music again and again
31
Invalidity
- X= Ew
- measuring the wrong construct reliability
- measure is reliable (can reproduce) but not valid bc we are measuring the wrong construct
- ex. test of research methods in latin
- measuring the wrong construct of Latin proficiency but reliable because can get the same grade again
32
Validity conceptual formula
- X=T
- measuring the desired theoretical construct/ the true construct
- because we have high validity= have high reliability
- because we are confident we are measuring the true construct then we can test the theory and hyp
33
Real world reliability and validity
- no measure is going to be perfecting reliable, going to have some Er
- no measure is going to be perfectly valid, going to have some Ew
- problem; don't know the actual amounts of T, Er and Ew in our observations therefore we have to estimate reliability and validity
34
Quantitative Index of Reliability (R)
- used to estimate reliability of measures
- largest when there is no random error; most reliable
- smallest when measure is entirely random error; no reliability
- reflects what proportion of X is meaningful and reproducible
35
Conceptual formula of R
R= T+Ew/ X
= T+Ew/ T+Ew+Er
- T and Ew are the reproducible and meaningful parts of the observation
36
Highest index of reliability
- Er=0
- R=1
- numerator will equal the denominator
37
Lowest Index of reliability
- X=Er
- R=0
- numerator is zero
- no reliable bc only made of random error
38
a good index of reliability ranges from
- 0-1
that is also the same range of the correlation coefficient. Therefore, correlation coefficients estimate the index of reliability
39
Internal Consistency
- extent to which individual items on a multi item scale tap into the same construct to produce similar scores
ex. want all items to measure depression
- responses to each item should highly correlate if measuring the same construct
40
Cronbach's Alpha
- also called Inter item correlation
- extent to which items correlate with eachother
- measure of internal consistency and assess reliability
- how closely related a set of items are
- reflection that participants are answering the items consistently and coherently
- ranges from 0-1
41
Cronbach alpha= 1
- highest internal consistency
- error free
maximal reliability
- confident we are measuring one construct
42
Cronbach alpha=0
- lowest internal consistency
- measure is entirely error
- no reliability
43
Test- Re test Reliability
- Administering the same test to the same individuals at 2 different points in time and correlating the scores
- if measuring the same underlying construct, expect the 2 scores to correlate highly
- if do not correlate, have to question if measuring a consistent construct
44
when can test re test reliability be used?
- only used when measuring smthg that you would expect to stay consistent in the sample
45
Inter Rater Reliability
- consistency across researchers
- useful when data is subjective and involves multiple raters
- measures the consistency bw 2 or more raters in their assessments, judgements or ratings of a particular phenomenon or behaviour
46
Higher agreement in inter rater means
- higher correlation and high reliability
47
lower agreement in inter rater means
- lower correlation and lower reliability
48
if data is quantitative and continuous use..
Pearson r correlation coefficient
49
Face Validity
- subjective way of gauging validity
- determining if it makes sense or if the measure appears to tap into the construct
ex. measuring sleeping makes sense to tap into the construct of depression, but measuring how much someone likes peanut butter does not make sense to tap into the construct of depression
50
Content Validity
- how well a test measures all relevant aspects of the construct it is designed to measure
- verifying that all major domains or aspects of the construct are being targeted in the measure
ex. depression has emotional, behavioural and cognitive elements. high content validity would tap into all of these. low content validity would only measure one of them
done by consulting with experts and literature
51
Criterion Validity
- how well a measure correlates with a specific outcome (criterion)
- more objective and involves a quantitative index
- extent to which observations from this measure relate to an outcome or criterion
Ex. scores on the GRE and success in graduate school would be in the index of criterion validity
52
2 types of Criterion Validity
- Predictive Validity
- Concurrent Validity
53
Predictive Validity
- relation bw a measure of a construct and a future criterion
- how well a measure taken at one time predicts a criterion in the future
ex. how do GRE grades predict 1st year graduate studies marks
54
Concurrent Validity
- relation bw the measure of a construct and the outcome at the same time
- predicting current outcome
ex. administering GRE to graduate students and see how it correlates to current grades
55
Construct Validity
- Extent to which constructs we claim to be studying are the actual constructs we are measuring
- are we truly assessing the construct of interest?
56
Face validity, content validity and criterion validity overall establish
- Construct Validity
57
2 types of construct validity
- convergent
- divergent/ discriminant
58
Convergent Validity
- scores of one measure correlate/ converge highly with scores of another measure of the same construct
ex. GRE and SAT are 2 difference measures of the same construct (scholastic aptitude) ^ should correlate
ex. my depression scale scores should converge highly with old depression scales
59
Divergent Validity
- scores on measures do not correlate highly with scores of measures of other constructs
ex. measure of depression should not correlate highly with political views
60
want to ensure
- scores of different measures of the same construct should correlate highly
- scores of different measures of unrelated constructs should not correlate
61
Do we use multiple estimates of validity?
- yes
- use multiple estimates of validity to gain as much confidence as possible we are measuring the construct of interest
