Lecture 2 - Dr Greg Yelland (DN) (incomplete) Flashcards Preview

z. Psychological Testing and Assessment > Lecture 2 - Dr Greg Yelland (DN) (incomplete) > Flashcards

Flashcards in Lecture 2 - Dr Greg Yelland (DN) (incomplete) Deck (29):
1

validity

- How well a test measures what it purports to measure

  • important Implications regarding
    • appropriateness of inferences made and
    • actions taken on the basis of measurements

 

 

2

precision

- sensitivity & specificity

- always a compromise between sensitivity & specificity

- usually screening process using sensitive test

- then use highly specific test to determine which actually have dementia

- 3:00

3

accuracy

- test needs to be accurate

6:30

4

reliability

- stability of measurement

- measurement is stable over time & within itself

7:20

5

what are the three components of reliability?

1) inter-rater reliability - more to do with scoring than the nature of tests

2) test-retest reliability - should get the same score when doing the same test twice

3) internal consistency - within the test ppl should be scoring consistently

- items should items should be equally good at measuring what they are trying to measure

7:50

 

6

What is test reliability?

- this is not scorer reliability

- test-retest - stability over time

 

- internal consistency

- homogenous - all items just testing one factor (anxiety)

- should be equally good at assessing that factor

- need to be aware of how many factors/behaviours a test is measuring

- if intend to measure one then should only measure one

10:00

7

What is reliability?

- the proportion of total variance (σ2) made up of the true variance (σ2tr)

- variability in test scores: σ2 = σ2tr + σ2e

- reliability of a test score is always made up of 

true score + error
X=T+E

- error is made up of random error & systematic error

8

Whenever we are talking about reliability & validity, we are talking about........

correlation or  correlation coefficients

- i.e.,  how well things are correlated on different aspects

e.g., with:

  • test-retest (looking at the correlation between first & second time test taken)
  • internal consistency (looking at the correlation between different items on the test)

15:30

9

What are some sources of error variance?

  • Test Construction
  • Test Administration

                  - Environment

                  - Test-Taker Variables

                  - Examiner-Related Variables

  • Test Scoring/Interpretation

each can contain both random & systematic error

16:20

10

What is the difference between systematic & random error variance?

  • Systematic - constant, or proportionate source of errror in variables other than the target variable
    • should not affect variance in scores
  • Random - caused by unpredictable fluctuations & inconsistencies in variables other than the target variable

 

Systematic changes should not affect the scores; unpredictable changes will affect the correlation; the more robust the test to fluctuation, the greater the reliability.

11

How does error occur in test contruction?

the way you select or sample test items

- if all items consistently perform in the same way (the way you intended them)

systematic error - could come from an ambiguous question - some ppl may respond one way and others another

random error - may have one or two questions where someone does not have enough experience to give the standard response to the item

 

17:00

12

How can error occur during test administration?

 Environmental Variables
 Test-Taker Variables
 Examiner-Related Variables

13

How do testtakers contribute to error?

Test-Taker Variables 

- during test administration

- differences between ppl taking the tests

systematic - different ages & not taking ages into account

random - age, personality etc

issue:

- dont necessarily want to minimize by only testing 10 year olds coz then test is only relevant to 10 yr olds

solution:

so do 10 yrs, 11yrs, 12yrs etc, then create norms for different ages (age norms) - takes care of the variable by having different normative data for different ages

20:00
 

14

How does the test environment contribute to error?

- during test administration

- one may be tested in noisy another in a quiet environment

- testing in a group or individually

affects test scores

 

15

How can examiners contribute to error?

- during test administration

- examiner humanness - may be exhausted by last test - may skip bits to hurry it up

16

How can test scoring/interpretation contribute to error?

- subjectively scored tests have greater error (because rely on subjective judgements)

- moving toward computer based scoring to remove this source of error

- cannot have computer based if its the quality of the response (qualitative) 

- much more error on qualitative than quantitative

 

22:35

17

What should we aim for with regard to error & reliability

aim to remove systematic error and minimise random error so we get better reliability

 

24:35

18

What are some reliability estimates?

  • test-retest
  • parallel forms/alternate forms

 

24:50

19

What is a test-retest reliability estimate?

24:45

  • same test taken twice - then see how well the scores are correlated
  •  issue of how long an interval between testing?

- the shorter the interval = the higher the test-retest reliability, because there are lots of things that can change in an individual over time

  • systematic changes should not affect test-retest reliability e.g., hot room, cold room (everyone affected equally) 26:50

- random changes will affect correlation (test-retest reliability) (27:15)

- the more robust the test is to fluctuation = more reliability

e.g.,  a test that is not affected by time of day, or amount of sleep etc - robust enough to wash those effects out - therefore (28:30)

  • participant factors will affect test-retest reliability - experience, practice, fatigue, memory, motivation, morningness/eveningness

                   - as everyone differs in these areas = greater error variance

                    - practise effects - give you a clue about what is going to happen next time we do the same test - this may mean that we cannot use test-retest 

24:45

20

When would we use Parallel or Alternate forms of a test?

- when we cannot use tes-retest reliability

- due to e.g., practise effects giving testtaker a clue about what will be on the test next time

21

What is a parallel forms or alternate forms reliability estimate?

  • parallel vs. alternate

             - parallel forms - are better developed

             - items have been selected so that the mean & variance has been shown to be equal

             - alternate forms - similar but no guarantee that variance is the same (hence have introduced a source of error)

  • testing is similar to process as test-retest - do one test then do the parallel or an alternate form.
  • test sampling issues - problem: is test sampling issue (choice of items)

             - best items are usually the best of the items available (unless create both tests at the same time

30:50

22

What is one of the biggest problems faced when using a parallel form or alternate form of a test? 

  • test sampling issues - problem: is test sampling issue (choice of items)

             - best items are usually used when creating the initial version of the test

 (unless creating both tests at the same time)

  • identifying source of error
  • is it because it is not stable over time or is it because the different items (content) of the two tests are introducing error

- is it stable over time? (external)

- internal consistency across the two tests? (internal)

33:50

23

Internal Consistency (Reliability)

- Split-Half testing

 Split into two halves
 Obtain correlation coefficient

24

What is the point of Split-Half testing?

 

 

 To obtain internal consistency of full version - Spearman-Brown Formula

Estimates internal consistency of a test that is twice the length

25

When is the Spearman-Brown formula used?

- To obtain internal consistency of full version - of split-half tests

- Estimates internal consistency of a test that is twice the length

 

- not used when more than one factor (heterogeneity)

- not appropriate for speed tests

- must have homogeneity when using split-half method because could end up with an imbalanced distribution of the factors across the two halfs

Spearman-Brown Split-Half Coefficient

rSB = 2rhh / (1+rhh )

rSB = 2 x 0.9/ (1+0.9)

rSB = 1.8/ 1.9


rSB = 0.947

26

When would we use Cronbach's Alpha?

- when we need an estimate to represent the sum of all of the individual variances in a split half test

- it estimates internal consistency for every possible split-half

- A generalised reliability coefficient for scoring systems that are graded by each item (sums all of them)

- used when items are graded (cannot not be used with dichotomous items)

- Essentially an estimate of ALL possible test-retest or split- half coefficients.

 α can range between 0 and 1 (ideally closer to 1)

- cannot measure mutliple traits - must be homogeneous

27

When would we use Kuder-Richardson?

51:25

- when test is dichotomous

- tests every possible split-half correlations or test-retest

- mainly used in split-half

28

What is acceptable range of reliability?

53:35

 Clinical – r > 0.85 acceptable
 Research – r > ~0.7 acceptable

 

Reliabilities of Major Psychological Tests
 

INTERNAL CONSISTENCY

  • WAIS – r = 0.87
  • MMPI – r = 0.84

TEST-RETEST

  • WAIS – r = 0.82
  • MMPI – r = 0.74

29

summary of reliability