assessment Flashcards
appraisal
process of assessing or estimating attributes. I.e. observations, survey, interviews
psychometrics
study of psych measurement
subjective format of test
relies mainly on scorers opinion, personal bias can impact rating.
halo effect
When a trait which is not being evaluated influences researchers rating on another trait. i.e. when someone is attractive, they may get higher rating in a test
objective test
raters judgment plays no or little role in scoring.
forced choice
items are known as recognition items (2 or more options). Person must respond with certain given answers (i.e. multiple choice)
difficulty index
indicates percentage of ind who answered each item incorrectly. From 0-1 to indicate percentage.
ipsative measures
measure compares traits within the same individual. Do not compare a person to other persons who took the same instrument. Person taking ipsative measure is measured in response to own standar of behavior. Measure points out highs and lows that exist within same ind. I.e. mr. johns depression is improving.
normative test
each item is independent of all other items. Can compare persons to each other who take same test
power test
evaluates level of mastery without a time limit. Can do power test even if it has time limit as long as time limit allows most people to complete test in time.
Achievement test: measures typical performance.
spiral test
Items get progressively more difficult (think of spiral staircase)
cyclical test
Several sections that are spiral in nature (each section gets harder as you go).
test battery
a horizontal test. Several measures are used to produce results that could be more accurate than those derived from merely using a single source.
vertical test
Versions for different developmental/education levels.
parallel forms (aka equivalent forms)
when a test has two interchangeable forms
most critical factors in test selection
validity and reliability
validity
whether the test measures what it says it does. The number one factors in choice selection. A valid test is always reliable.
5 types of validity
Content (aka rational/logical): Does the test sample the behavior under scrutiny. I.e. IQ test that didn’t sample entire range of
Construct: Ability ot measure a theoretical construct like intelligence, talent, etc. Any trait you cannot measure or observe is a construct.
Concurrent: How well test compares to other well established instruments with same purpose.
Predictive: Ability to predict future behavior.
Consequential: social implications of using tests.
convergent validity
Relationship/correlation of a test with an independent measure or trait.
discriminant validity
test will not reflect unrelated variable
criterion validity
could be predictive or concurrent
face validity
the extent that a test looks or appears to measure the intended attribute
incremental validity
describe process by which a test is refined and becomes more valid as contradictory items are dropped. Test’s ability to improve predictions when compared to existing measures. Test provides additional valid information that was not attainable via other procedures
synthetic
Derived from “synthesized”. The helper or researcher looks for tests that have been shown to predict each job element or component, then combines these tests to create one with synthetic validity.
reliability
how consistent a test measures an attribute. Second most important concern in test selection. Tests can be reliable, but not valid.
reliability coefficient
0-1 (1= perfect and only occurs in physical measurements). In psychological tests, 0.90 is considered excellent. 0.70 is considered somewhat typical. Tells you the percentage of the score that is accurate.
.80 is considered acceptable for admissions to jobs, etc.
true variance aka coefficieint of determination
the square of the correlation/reliability coefficient (i.e. if correlation is .7, the true variance is .49 or 49%).
test-retest reliability
giving the test to the same group of people two times and correlating the scores.
equivalent or alternative forms of reliability
Giving the same population alternate forms/parallel forms of the identical test. Reliability correlation coefficient is found on the two sets of scores.
split-half correlation method
Dividing a test into two halves and then finding the correlation between the half scores to find the reliability coefficient.
inter rater/inter observer method
aka “scorer reliability” . Used with subjective tests to see if the scoring method provides same score despite different scorers. Several raters assess the same performance.
consistency reliabilty
homogeneity or internal consistency or inter-item consistency.
counterbalancing
Necessary practice for equivalent forms reliability testing. Half individuals get parallel form A first and half get form B, then switch for the second.
stability
the ability of a test score to remain stable or fluctuate over time if the client takes the test again