Criterion Related Validity Coefficient
Coeffecient representing usefulness of predictor for predicting criterion outcomes; uses correlation. Higher correlation means higher test scores = better performance. Increased confidence in selection procedures.
Taylor and Russell use of validity coefficient
Usefulenss depends on selection ratio and base rate of success.
- •Taylor & Russell (1939) showed that a selection test is most useful when validity is high and selection ratio is low
•If selection ratio is high, test doesn’t matter much (you hire people anyway)
•If validity is low, test won’t help you (even if you are very picky)
Number of openings per number of applicants
Base rate of success
Unscreened employees who are currently successful on the job
When is validity coefficient not useful
When the base rate of success is very high or very low.
A correlation that reflects the consistency and stability of a measure, over time, across equivalent forms, or among internal items.
Refers to the additional predictive validity that can be gained by adding more measurement devices to an assessment program.
Refers to the extent to which a validity established in one setting with one sample can be generalized to another setting and sample.
A legal term that refers to the potentially unintentional discrimination against a protected group (usually minority racial groups and women) due to the use of a selection or placement procedure. (less than 80%)
Reliability coefficient - calculation
Used to assess the consistency or a measurement of instrument or procedure, internally, with different forms, or across time.
Reliability Coefficient - statistic
Computed to asses the consistency in judgment of 2 or more human raters engaged in the same rating task. (i.e., test-retest, equivalent-form, split-half, inter-rater.
Coefficient of stability, measurement given to same group at two different times, risk of contamination from carryover of first test to second test.
Coefficient of equivalence, parallel forms of the instrument are given to the same group at the same time; difficulty generating two forms
Split-half or Chronbach's alpha; correlate first half to second half or correlate every other item; widely used in I/O
Correlate scores of various raters; important in I/O for job analysis ratings and employment interviews
Quality Assurance (QA)
Planning and processing moves undertaken by an organization to ensure products/services serve intended purposes (which are defined by users of the product or services.
Process to ensure QA
Assmts conducted over time to determine reliability; controlling and monitoring materials, processes, mgmt., and personnel.
Quality Control QC
Finding defects/errors before customer presented with product; based on organizational standards and expectations, not external feedback; QC can inform QA process
Total Quality Management TQM
Attempting to embed QA in all aspects of organization; assigning responsibility for QA to organization's production, process, and distribution members. Improving quality and meeting demands.
Methods & procedures used to evaluate & understand individuals in the workplace. That outcome is used to make decisions about individual performance, skill and training levels, aptitude, as well as organizational processes.
Bxs performed by an employee that are not part of his or her regular tasks, but that reflects positively on the organization or otherwise assist the organization. AKA organizational citizenship.
What happens when selection ratio is high
This indicates that most applicants will be hired, including those that will be unsuccessful.
What happens when selection ratio is low
Indicates few applicants hired per number applied, minimizes inclusion of unsuccessful candidates
What happens when base rates are high
Indicates that most employees are successful, thus most candidates will be as well and the test won't be necessary or helpful.
What happens when base rates are low
Indicates the test won't discriminate those who won't be successful
What happens when base rates are close to .50?
Validity coeffecients will be best able to determine success of the candidate.
What is the best scenario regarding selection ratio?
Selection ratio is low, base rate is moderate, then even lower validity coefficients are informative.
Ways to mitigate adverse impact
Separate cutoff scores for different groups, banding, selection of predictors determined to be valid and reliable.
using ranges of scores where all scores within a range are considered equivalent
Occurs when the assessment on the conceptual criterion (e.g., job performance) is made not only by relevant actual criteria measures, but also in terms of actual criteria that could be considered extraneous.