How concepts are measured in research

Question 1

define operationalisation

Answer 1

the process of turning a concept into something that we can measure. It involves a researcher thinking about, and then deciding on, how they will obtain numeric information about a concept.

Question 2

what makes operationalisation difficult

Answer 2

Ideally we can (simply) observe the concept - ‘direct measurement’
But many concepts in education and in psychology cannot be measured directly - their operationalisation is more complex that simple direct measurement via direct observation

Question 3

what is direct measurement of a concept?

Answer 3

concepts that we can directly observe
there is little to no operationalisation

Question 4

example of direct measurement

Answer 4

household income

Question 5

how is direct measurement depicted graphically?

Answer 5

rectangles used for directly observable concepts and/or their causes or consequences

Question 6

what is indirect measurement of a concept?

Answer 6

concepts that we cannot directly observe
operationalisation of the concept occurs by observing its causes and/ or consequences

Question 7

example of indirect measurement

Answer 7

student motivation

Question 8

how is indirect measurement depicted graphically?

Answer 8

rectangles used for directly observable concepts and/or their causes or consequences
ellipses used for non-directly observable consequences

Question 9

how can indirect measurement be broken down?

Answer 9

reflective measurement and formative measurement

Question 10

define reflective measurement

Answer 10

We operationalise our concept by its consequences across a range of things that we can observe
more common in education and psychology

Question 11

why is a range used?

Answer 11

to avoid confusing the hidden concept with one of its observable consequences

Question 12

define formative measurement

Answer 12

We operationalise our concept by its causes across a range of things that we can observe
much rarer in education and psychology
Most commonly used when the non-directly observable concept is a risk of something negative e.g. dropping out of school due to illness or bullying

Question 13

what questions must a researcher ask when it comes to measurement?

Answer 13

1. Whether a concept can be directly or indirectly measured and how?
2. The level of detail about a concept that is required? What level of measurement should be used?
3. What measurement process to use to get the required numeric information? What research methods can be used to get information on a concept

Question 14

what are examples of research designs?

Answer 14

case study
survey
experiment

Question 15

what are examples of research methods?

Answer 15

questionnaire
interview
observation
test

Question 16

define validity and reliability

Answer 16

Validity refers to accuracy
Reliability refers to consistency
Validity partly depends upon reliability

Question 17

define construct validity

Answer 17

Construct validity is how accurate a measure is at representing a concept

Question 18

what are the two ways to show construct validity?

Answer 18

1. Convergent validity - data from measures that are supposed to be related due to theory, are in fact related
   
   2. Discriminant validity - data from measures that are not supposed to be related, are not in fact related

Question 19

what are the three types of measurement reliability and its definitions?

Answer 19

1. Stability - consistency over time
   
   2. Inter-rater reliability - consistency across different people
   3. Internal consistency - when we try to infer a concept based on many directly observed measures

Question 20

measurement error and bias

Answer 20

No measure is 100% accurate/valid - always a small degree of error
We make our measures as accurate as possible by minimising systematic measurement error

Question 21

define systematic measurement error

Answer 21

errors in the process that can be controlled and causes bias

Question 22

define random measurement error

Answer 22

errors in the process that cannot be controlled

Question 23

common sources of measurement bias

Answer 23

• People
• Research methods/ instruments
• Procedures
• Environments

Question 24

common consequences of measurement bias

Answer 24

• Some groups of participants are likely to not have a response
  
  • The measurement may be inaccurate/ invalid
  • May be a taboo subject to some people causing people to respond differently

Question 25

items and scales

Answer 25

When we measure a concept we do so by combining lots of measurements This occurs not only with inferred measurements but also direct measurement

Question 26

define items

Answer 26

refers to the measures that we will combine

Question 27

define scale (two definitions)

Answer 27

refers to the result of the combination of these measures/items used to describe the range of possible responses to an item

Question 28

what are the three ways to construct a scale from items?

Answer 28

1. Summation - add up items to make the scale 2. Average - mean, median, mode 3. Factor analysis - a statistical process whereby a number of scales and their relationship to items are estimated based on the results of a statistical analysis

Question 29

what is a likert scale?

Answer 29

these are items with a given set of response options. These reponse options have a specific set of words. The words result in an ordinal measure e.g. strongly agree; agree; neither agree nor disagree; disagree; strongly disagree