lecture 6 - data and measurement Flashcards
types of data
primary (collected by researcher) vs secondary (collected by someone else)
quantitative (numeric) vs qualitative (words/texts/images/videos)
by source:
- people
- observation
- documents
*incl. big data, although this is hard to classify - secondary sources
unit of analysis + fallacies
= the entity of that what is being studied (e.g. countries, individuals, treaties, policies)
unit of analysis -> characteristics/attributes -> measurement
- e.g. country -> size, democracy -> statistics, ratings
- e.g. individual -> age, attitude -> survey
fallacies:
conclusions need to stick to the level of analysis you chose
- ecological fallacy = when you study macro-level but make micro-level conclusion (e.g. observe that a country is rich, conclude that all citizens ar rich)
- individualistic/exception fallacy = study micro-level -> macro-level conclusion
aka overgeneralization
concept
= constructs derived by mutual agreement from mental images that summarize collections of seemingly related observations and experiences
- usually have multiple attributes/dimensions/indicators
*ideally: interchangeability (leaving one indicator out would/should still correctly identify the concept)
theory, measurement real world
theory -> concepts = conceptualization
measurement -> indicators and variables = operationalization
real world -> phenomena = observation
Gerring: conceptual goodness
- familiarity (estabished usage)
- coherence (internal consistency)
- resonance (cognitive click, needs to make sense to people)
- parsimony (simple and clear)
- depth (ability to bundle many characteristics/attributes)
- differentiation (external boundedness/differentiation/distinguishing concepts from one another)
- theoretical utility
- field utility
= trade-offs
operationalization
= translating concepts into something that can be observed (indicators)
*needs to reflect all elements of the conceptualization
examples
- gender: in surveys multiple options: male/female (conceptualization as biological sex) vs male/female/no comment/neither (fits with conceptualization as social construct)
- corruption: operationalization has political, economic and legal elements
*concept = misuse of public position of private gain
*indicators = perception of corruption by business people + experience of corruption by public + prosecution of public officials
*observations: expert survey + public survey + court records
measurement should be
- unbiased: free of systematic errors
- efficient: low variance/random errors
types of measurement validity
measurement reliability = accuracy
face validity = judgment based
content validity = theory-based (does a measure cover all elements the theory requires)
criterion/construct validity: criterion-based
- concurrent and predictive validity
- convergent validity: different measures of the same thing should correlate
- discriminant validity: thing should only measure the specific element it is supposed to measure
*it should not explain everything, e.g. verbal test should not predict results in a math test
types of measurement reliability
- stability over time = consistency and precision of results
- consistency across indicators = internal consistency
- consistency across researchers/judges = intercoder/inter-rater reliability
!validity is more important than reliability: if you have reliability you have nothing meaningful yet
measurement error
- random error = not systematic, can make for small deviations -> more unreliable and unprecise, but not wrong
- systematic error = never gets the right measurement = invalid and biased
measurement reliability: coefficient
= quantitative measure for internal consistency
range 0 (non) - 1 (perfect) = 0%-100% internal consistency
- rules of thumb:
.70 = minimum
.80 = desirable
examples of reliability coefficients:
- Cronbach’s a(alpha) = correlation of all indicators/items of a multi-item scale
- split-half method: split and combine indicators intwo two sets/measures and correlate them (see if they measure the same thing)
triangulation
= repeat a study to see if it is robust
3 types:
- data triangulation
- investigator triangulation
- methodological triangulation
3 possible outcomes:
- convergence
- inconsistency
- contradiction
Mathison: inconsistency is good, you can learn from it
- triangulation does not solve the measurement problem, it is a learning process to in the end lead to better measurement
data quality
proff: quality over quantity
quality lies in:
- transparency
- replication
- verification
