Lecture 3 - Test Development 1: part2 - Construction (DN) Flashcards Preview

z. Psychological Testing and Assessment > Lecture 3 - Test Development 1: part2 - Construction (DN) > Flashcards

Flashcards in Lecture 3 - Test Development 1: part2 - Construction (DN) Deck (22):

What is the difference between scaling & measurement?

  • Measurement: the assigning of numbers to some attribute
  • Scaling: the method (mechanism) by which we assign numbers (rule setting)


What types of rules guide measurement?

  • Nominal rule
    • classifies only
  • Ordinal rule
    • slightly more sophisticated
    • it is ordered
    • train example - order says nothing about distances between stations
    • has both nominal & ordinal properties
  • Interval rule
    • has nominal & ordinal properties
    • equal distance between
    • no absolute zero
    • zero is just arbitrary
    • e.g., celsius (zero = freezing, but not the absence of tempretaure)
  • Ratio rule
    • true zero point
    • not a lot in psych testing



Which rule would apply if the following exist

  • Categories
  • Ordinal
  • Equal Intervals
  • Absolute Zero Point

Ratio Rule


Which rule would apply when the following exists

  • Categories

Nominal rule


Which rule would apply if the following exist

  • Categories
  • Ordered (ordinal)
  • Equal Intervals

Interval rule



Which rule would apply when the following are present

  • Categories
  • Ordering

Ordinal rule


What level of scale would you require if you are wanting to get an aggregate (summative) score?

i.e., Summative Scale

  • At least Interval



What level of data is produced when using a likert scale?


  • ordinal-level data
  • because they do not actually have equal levels between responses
    • however they are treated as  interval (summative) scale
    • because they behave interval like (although theoretically there are not equal distances between strongly disagree & disagree)


What level of data is derived from binary choice scale?

  • nominal
  • not as much info as a likert,  but its simpler




Why would you use a faces scale?

  • good for children
  • they dont understand things in the same way adults do




What are some other scaling methods that yield ordinal data?


  • method of paired comparisons (1:17:19)
    • must choose one on basis of some rule
  • comparative scaling (1:19:19)
    • sorting/ranking along a dimension
    • what is worse cheating on taxes or cheating on test
  • categorical scaling (can be nominal data)
    • sorting stimuli according to rule
    • good or bad / best or worst
    • essential - not essential (our questionnaire)
    • depends on how question is framed as to whether you get ordinal or nominal data (with categorical)


What are the main item formats used when writing test items?


  •  Constructed-response format
    • 1. Completion item
    • 2. Short-answer
    • 3. Essay


  • Selected-response format
    • 1. Matching items
    • 2. True-False
    • 3. Multiple choice


What are the different types of  constructed-response format?

1. Completion item (1:25)

  • provides word to complete sentence
  • must be a specific correct answer
  • poor design can lead to scoring problems


2. Short-answer (1:31:15)

  • word, sentence, paragraph in answer to a highly specific question


3. Essay (132:10)


What are the different types of selected-response formats?

1. Matching
2. True-False (1:34:20)

  • must contain a single, unambigous idea & be short
  • use kuder-richardson for binary

3. Multiple choice (1:35:35)

  • a stem
  • a correct item
  • multiple incorrect alternative options (distractors, foils)
  • stem - expectancy tables are used to evaluate this
    • could use kuser-richardson & look at it internally
    • or look at all 4 and use chronbach alpha


What are the four complex multiple choice formats?


  • Classification
  • If-then conditions
  • Multiple conditions
  • Multiple true-false
  • Oddity


Which scaling methods did we use & why?


Previous research (Talbot et. al.)

  • used Likert (ordinal)
  • made it summative (assumed interval) to get a composite score


We will use 

  • Likert (assume interval)
    • how felt if not enough sleep
  • Categorical
    • emotion is essential, useful, unnecessary
    • to behaviour of whether you get enough sleep


What is an If-then condition?


  • one of four complex multiple-choice format
  • The test-taker must decide the consequence of one or more conditions being present

If the true variance of a test increases but the error variance remains constant, which of the following will occur?”

a. Reliability will increase

b. Reliability will decrease

c. Observed variance will decrease

d. Neither reliability nor observed variance will change

if variance has an increased proportion of true variance - this will always increase reliability


What is meant by the term multiple conditions in terms of item format?


  • it is one of the four complex multiple-choice formats
  • The test-taker uses two or more conditions in the statements listed to draw a conclusion

“Given that Mary’s raw score on a test is 60, the test mean is 59, and the standard deviation is 2, what is Mary’s z score?”

a. -2.00

b. - .50

c. .50

d. 2.00

z score = difference between raw score & mean / standard deviation

1 / 2 = .50


What are multiple true-false items?


  • one of four complex multiple-choice formats

The test-taker decides whether one, all, or none of the two or more conditions or statements listed in the stem are correct

“Is it true that (1) Alfred Binet was the father of intelligence testing and the (2) his first intelligence test was published in 1916?”

a. Both 1 and 2
b. 1 but not 2
c. 2 but not 1
d. Neither 1 nor 2


What is an oddity item?


  • one of four complex multiple-choice formats
  • The test-taker indicates which option does not belong with the others
    • which is the odd one out?

“Which of the following names does not belong with the others?”

a. Alfred Adler

b. Sigmund Freud 

c. Carl Jung

d. Carl Rogers

Freud, Jung & Rogers = listeners

Adler = confronter


What is the first step in writing test items?

  • create an item bank or pool
  • start off with at least twice as many as you want to end up with


What is a classification item format?

  • one of four complex multiple-choice formats
  • The test-taker classifies a person, object or condition into one of several categories designated in the stem

Jean Piaget is best described as a _____________ psychologist

a. Clinical

b. Developmental 

c. Psychometric

d. Social