Chapter 5 Reliability

Question 1

It refers to consistency in measurement, it only really refers to something that is consistent–not necessarily good or bad but simply consistent

Answer 1

Reliability

Question 2

an index of reliability, a proportion that indicates the ratio between true score variance on a test and the total variance.

Answer 2

Reliability Coefficient

Question 3

An index describing the consistency of the scores across context

Answer 3

Reliability Coefficient

Question 4

This theory states that a score on an ability test is presumed to reflect not only the test taker’s true score but also the error

Answer 4

Classical Test Theory

Question 5

a portion of our observe score which extent of our ability, characteristics and behavior

Answer 5

True score

Question 6

the component of the observed test score that does not have to do with the test taker’s ability

Answer 6

Error

Question 7

It is a statistic useful in describing scores of test score variability. It is useful because it can broken down into components

Answer 7

Variance

Question 8

What are the two components of variance?

Answer 8

True variance and Error Variance

Question 9

It is a variance from true differences

Answer 9

true variance

Question 10

Variance from irrelevant various sources

Answer 10

error variance

Question 11

The greater the proportion of the total attributed to true variance, the more____is a test

Answer 11

reliable

Question 12

all factors associated with the process of measuring some variable, other than the variable being measured;

Answer 12

measurement error

Question 13

It is a source of error caused by unpredictable fluctuations and inconsistencies of other variables in the measurement process. It is also called “noise”

Answer 13

Random error

Question 14

A type of error that is typically constant or proportionate on what is presumed to be the value of the variable being measured

Answer 14

systematic error

Question 15

True or False. Is Systematic error can be fixable once it is discovered?

Answer 15

TRUE

Question 16

True or False. Is Systematic error does affect the score consistency?

Answer 16

FALSE

Question 17

According to this theory, we can estimate the true score by finding the mean of the observe scores from repeated administration

Answer 17

Basic Sampling theory

Question 18

Which source of error is this situation? The Extraversion Personality test constructed by the students of Ms. Salinas has a variation among items within a test and variation among items between items

Answer 18

Test construction under item/content sampling

Question 19

What are the three sources of error under test administration?

Answer 19

Test-environment, test-taker variables, and examiner-related-variables

Question 20

A source of error in which the sample was actually represent the population but it is not enough

Answer 20

Sampling error

Question 21

A sources of measurement error in which a variability inherent in a test score as a function of the fact that they are obtained at one point in time rather than another. Same test given at different points in time may produce different scores, even if given to the same test takers tests of relatively stable traits or behavior may be prone to this error

Answer 21

Time Sampling

Question 22

A sources of measurement error that results from selecting test items that inadequately cover the content area that the test is supposed to evaluate

Answer 22

Item/content sampling

Question 23

A sources of measurement error that is concerned with the intercorrelations between items in a test.

If the test is designed to measure a single construct and all items are equally good candidates to measure that attribute, then there should be a high correspondence among items

Answer 23

Inter item inconsistency

Question 24

A sources of measurement error in which different judges observing the same event may record different numbers

Answer 24

Observer Differences:

Question 25

A reliability estimate in which it correlates pairs of scores from SAME people who are administered the SAME test at two DIFFERENT times.

Answer 25

Test-Retest Reliability Estimate

Question 26

what is the result of time sampling error in test-retest reliability

Answer 26

scores are likely to fluctuate as a result of time sampling error longer interval = lower correlation

Question 27

What is the ideal interval of re-administering a test in test-retest reliability?

Answer 27

(ideal interval between tests is 2-4 weeks

Question 28

What are the statistic procedure use in test-retest reliability?

Answer 28

statistics: Pearson r -> interval and ratio Spearman rho -> ordinal

Question 29

What are the possible intervening factors that will happen if we did not follow the interval of 2-4weeks of re-administration of a test in a test-retest reliability?

Answer 29

carry over effect Practice effect Mortality Changes in Participants Combination of all these factors

Question 30

explain the carry over effect in test-retest reliability

Answer 30

--possible na maalala yung tests at mag review occurs when the first testing session affects the second testing (e.g. remembering test items) only of concern when it is random or if it is unpredictable and affects only some respondents if systematic error, where it affects all respondents, then reliability is not affected

Question 31

explain the Practice effect in test-retest reliability

Answer 31

when a test takers score better because they have sharpened their abilities with the passing of time—-- development shows plasticity hahaha

Question 32

explain the coefficient stability

Answer 32

Kapag naman masyadong matagal 6 months for example wala nang masyadong maalala si test taker–ang tawag pag ganito is Coefficient of Stability

Question 33

Explain Mortality in test-retest reliability

Answer 33

dropping out of the study.

Question 34

Explain changes in Participant in test retest reliability

Answer 34

-non normative changes and normative history graded influences

Question 35

It is the degree of the relationship between various forms of a test that can be evaluated by means of alternate forms of a test or parallel forms coefficient of reliability

Answer 35

Coefficient of equivalence

Question 36

In this reliability of estimate, the means and variances of observed test scores are equal

Answer 36

Parallel forms

Question 37

In theory, the mean scores obtained on ______ correlate equally with true score and with other measures

Answer 37

Parallel Forms

Question 38

A reliability estimate in which simply different versions of a test that have been constructed to as parallel. It is a mere different versions without the sameness of the observed scores

Answer 38

Alternate Forms

Question 39

It is typically designed to be equivalent with respect to variables such as content and level of difficulty

Answer 39

Alternate Forms

Question 40

How to obtain the estimates in parallel/alternate forms

Answer 40

- two administrations with the same group are required (Form A at one point and Form B at the other) - the equivalent form is administered either immediately or fairly soon -after administration, correlation coefficient is obtained between the results of the two forms

Question 41

What are the statistic procedure that we can use in Alternate or Parallel forms?

Answer 41

- statistics Pearson r for scale measurements (interval/ratio) - Spearman rho for ordinal

Question 42

Source of error in immediate Alternate/Parallel forms

Answer 42

content sampling

Question 43

Source of error in delayed Alternate/Parallel forms

Answer 43

content sampling and time sampling

Question 44

True or False. Alternate forms is are INDEPENDENTLY CONSTRUCTED TESTS

Answer 44

True

Question 45

Give the following that alternate and parallel forms should have

Answer 45

- the same number of items - items expressed in the same form - items that cover the same type of content - same difficulty - same instructions -same time limits, format, and all other aspects of the test

Question 46

degree of correlation among all the items on a scale calculated from a single administration of a single form of a test

Answer 46

Internal Consistency

Question 47

Explain the term "assess homogeneity" in Internal consistency

Answer 47

extent to which items measure a single trait

Question 48

True or False. Is possible that heterogenous items can be homogenous

Answer 48

True

Question 49

Explain why is it possible that heterogenous items can be homogenous

Answer 49

possible pa rin naman kahit heterogenous yung items basta i-measure ang items per subscale Homogenous pa rin ang test na may subscales as long as per subscale iisang construct lang yung minemeasure.

Question 50

In this reliability estimates, the two scores are obtained for each person by dividing the test into equivalent halves

Answer 50

Split-Half Reliability Estimate

Question 51

What are the steps of Split-Half Reliability Estimate?

Answer 51

- divide the test into two equivalent halves - calculate Pearson r between scores on the two halves of the test - adjust half-test reliability using the Spearman-Brown formula

Question 52

What are the acceptable ways to divide the test in Split Half Reliability?

Answer 52

- random assignment - odd-even - split dividing the test by content so that each half contains - equivalent items with respect to content and difficulty

Question 53

Explain the disadvantage of split half reliability

Answer 53

reliability of the test is directly related to the length of the test, kaya pag split half yung ginamit mo possible na below .70 yung correlation ng scores. To address that gagamit ng spearman brown formula. Rule of thumb is the higher the number of items the higher the reliability

Question 54

What is spearman brown formula?

Answer 54

- it is used in split-half tests in correcting their correlation - estimates the internal consistency reliability from two halves of a tests - can be used to determine the reliability of one test once it is shortened or lenghtened

Question 55

What are the other functions of spearman-brown formula?

Answer 55

- it can determine the number of items needed to attained a desired level of reliability

Question 56

In adding new items using spearman brown formula, what are the factors to consider

Answer 56

it must be equivalent in content in difficulty so that the longer test still measures what the original test measures

Question 57

if the reliability is low in the context of split half and spearman brown, what we can do?

Answer 57

- abandon the instrument - locate or develop a suitable alternative - create new items, clarify the test instruments or simply the scoring rules

Question 58

used for non-dichotomous items —walang maling sagot na test such as personality tests

Answer 58

Cronbach’s Coefficient Alpha

Question 58

True or False. It is appropriate to use split half reliability in heterogenous test and speed test

Answer 58

False

Question 59

According to Steiner, the value of alpha____ may be too high and indicate redundancy items

Answer 59

.90

Question 60

Cronbach's alpha will be higher when a measure has more than ___items

Answer 60

25

Question 61

used for tests with dichotomous items with varying difficulty May choices at may iisang tamang sagot lang na test and magkakaiba yung level of difficulty per item

Answer 61

KR-20 (Kuder-Richardson formula 20)

Question 62

who is the propent of

Answer 62

KR-20 (Kuder-Richardson formula 20)

Question 63

used for tests with dichotomous items with same difficulty or average 50% difficulty

Answer 63

KR-21

Question 64

A relatively new measure for evaluating internal consistency of a test. It focuses on degree of differences that exist between item scores

Answer 64

Average Proportional Distance (ADP)

Question 65

Interpretation of ADP

Answer 65

0.2 higer---excellent .25 to 0.2 Acceptable

Question 66

ADP with .25 measure means

Answer 66

it suggests that there is a problem in internal consistency of the test

Question 67

What is one potential advantage of APD to Cronbach's Alpha?

Answer 67

APD is not connected to the number of items on a measure

Question 68

A reliability estimate that focus on degree of agreement or consistency between two or more raters with regard to a particular measure

Answer 68

. Inter-rater reliability

Question 69

. Inter-rater reliability is often use in _____

Answer 69

evaluating non-verbal behavior

Question 70

what is the source of error in Inter-rater reliability?

Answer 70

differences between raters assessed / scoring

Question 71

What are the statistical procedure use in inter-rater reliability

Answer 71

Kappa Statistics: 2 raters Fleiss Statistics: 3 rater

Question 72

If the test is designed for use at various times or for stable traits (e.g. employee performance, and personality trait, then what reliability estimate should we use

Answer 72

Test-retest reliability

Question 73

If the test is for single administration only. then what reliability estimate should we use?

Answer 73

Internal Consistency

Question 74

____ is a source of error attributable to variations in the test takers feelings, moods, or mental state overtime

Answer 74

Transient Errors