Week 6 Flashcards
Test Administration, Norms & Standardised Scores (26 cards)
What is the norm group?
The representative sample of examinees whose score will be obtained so future tests can refer it to
Why is test standardisation done?
Test standardisation is done to establish the distribution scores in the norm group so that future examinees can be compared with that distribution
What are the stages in norms development?
- Developing the test
- Define, create and analyse the items
- Obtained evidence of validity and reliability - Sampling
- Find a sample of people who are representative of the population the test is intended for - Testing and Scoring
- Ask the representative sample of people to answer the test and obtain a raw score for each examinee - Describing the Scores
- Prepare central tendency, variability and form of distribution stats for the raw scores - Standardising
- Select a standardisation system and transform the distribution of raw scores into such system
What is the developmental norm of Age Equivalence?
Depict the level of test performance for each separate age group in the normative sample
Try to pick up the differences in performance on a test that is associated with the development/ageing process.
Age determined - Compared to age group
What is the developmental norm of Grade Equivalence?
Depict the level of test performance for each sample grade in the normative sample.
Usually achievement tests to determine whether an individual is performing at a lower, equal or greater level in comparison to others from the same grade.
Grade determined - Compared to grade group
What are the 3 approaches to sampling?
- Random Sampling
- Stratified Random Sampling
- Good Faith
What is the random sampling approach?
Each member of the population has an equal chance of being selected to be part of the norm group
Really difficult, near impossible to achieve and rarely done
Hard to obtain consent and replacing them would also prove difficult → sample that is not randomly sampled
What is the Stratified Random Sampling approach?
Classifying the population on important background variables.
Selecting an appropriate proportion of people at random from each stratum.
E.g., Population is 60% composed of females so you try and ensure that your standardised sample/norm is 60% composed of women etc.
Repeat for other notable variables e.g., Age → 30% of population is aged 60+ so the standardised sample has to be composed of 30% of individuals who are over 60.
You look at census data of population and then recruit based on that information
Census → Recruitment
What is the Good Faith approach to sampling?
Effort to select a diverse sample of people who are representative of the population.
Compare to relevant census
Least desirable but highly cost and time effective
You recruit and then compare it to census data on the population
Recruitment → Census
E.g., The population may be composed of 60% women but in your standardised sample/norm group, women only comprise 40% of it.
What are 4 methods to describe group performance?
- Central tendency
- Variability
- Distribution Stats
- Distribution Graphics
What are the SD values for 68%, 95% and 99% in a normal distribution
68% = 1 SD, 95% = 1.96 SD, 99% = 2.58 SD
What is percentile and percentile rank?
Percentage of persons in the standardisation sample who scored below a specific raw score.
Denoted with P.
P25 = First Quartile (Q1) = 1 quarter of scores are below this value
P50 = Second Quartile (Q2) = 2 quarter of scores are below this value (Median)
P75 = Third Quartile (Q3) = 3 quarter of scores are below this value
How do you calculate percentile?
Arrange the data from smallest to largest
Decide which percentile you want to calculate
Pi = (npi / 100) + 0.5 Where: Pi = the percentile you are interested in n = the total number of scores in the sample.
How do you calculate percentile rank?
What percentage of the scores fall below a particular score (Xi)
Pr = (B/N) * 100 Where: Pr = Percentile Rank Xi = the score of interest B = the number of scores below Xi N = the total number of scores
What are the differences between percentile and percentile rank?
Percentile:
- Indicate a position in the distribution that corresponds with a specific raw score
—> Ordinal Placement, Ordinal Position
Percentile Rank:
- Indicates the percentage of cases falling below a given raw score
—> Percentage of sample they scored above
What are the Pros and Cons of Percentiles and Percentile Rank?
Pros
- Easy to calculate
- Intuitively appealing
Cons
- They distort the underlying measurement scale –> no information magnitude between ranks
- Often confused with % of correct answers
What are some problems with percentile?
It does not preserve the properties of the original scores
Same distances in raw scores will not correspond with differences of equal magnitude in percentiles
- Ordinal
Example
Between -2SD and -1SD there is a 13.59% difference in percentile, however,
Between +2SD and +3SD there is a 2.14% difference in percentile
What does it mean by “Non-Linearity of the Percentile Transformation”
Interpretation of percentile ranks is simple and straight-foward
However, the percentile transformation has limited usefulness in data analysis as it does not preserve the equivalence of distances between raw scores.
- The real difference between raw scores is minimised near the ends of the distribution and exaggerated in the middle of the distribution.
– Between percentiles 2 and 4, there is only a raw score difference of 1 which is the same difference between percentile ranks of 40 and 47.
What does it to transform raw scores into standard scores?
Represent distance from the mean in standard deviation units
Standard scores retain the relative magnitude of differences between values in the raw scores
What are the characteristics of Z-score and how to calculate it?
Z = (X - X’)/ S
X = individual Score
X’ = Mean
S = Standard Deviation
Characteristic of Z scores
Mean = 0
SD = 1
Can be positive or negative
What is the standardised score system formula?
Conversion of Z-scores into a new system with an arbitrarily chosen mean and standard deviation.
Generic Formula to transform Z-scores into other system:
y = SZ + M Where: y = Score in target system S = Standard Deviation of the target system Z = Specific Z-score to be transformed M = Mean of the target system
What is the mean, SD and formula of McCall’s T
Mean always 50 and SD always 10.
T = 10*Z + 50
What is the mean, SD and formula of IQ scores
Mean always 100 with SD always 15
IQ = 15*Z + 100
What are the pros and cons of standard scores?
Pros
- Allows for comparison of individuals in same test and across tests
- Many traits appear normally distributed, hence amenable to standardisations
- Maintain magnitude of difference in raw scores
Cons
- Requires knowledge of normal curve and z-scores
– Not understood by general public
- The Mean and SD of different systems varies, which can be confusing
