According to the authors, the best way to test people’s writing ability is....
to get them to write (test writing ability directly)
Writing task problems
- Representative Tasks (population + expected performance)- Valid samples of writing (represent ability)- Scored Validly (Can/Will be)
How to Create Representative Writing Task
- Specify all possible content- Include Representative Sample of Specified Content
Specify all possible content of a Writing Task
Operations (Describe, explain, compare, etc.)Types of textAddresseesLength of textsTopicsDialect and style
Describe, explain, compare and contrast, argue for and against a position, etc.
ideal writing test for Construct Validity
One which requires candidates to perform all the relevant potential writing tasks.
Why writing task with- Wide ranging - Representative sample of specifications
Really good coverage of the range of potential writing tasks is...
Not possible in a single version of the test.
To know something ACCURATE and MEANINGFUL about a person’s WRITING ability is ALWAYS
Elicit a valid sample of writing ability
- Separate tasks (as many as is feasible)- Test ONLY writing ability- Restrict candidates
Ensure valid and reliable scoring
- Reliably scored tasks- As many as possible tasks- Restrict candidates (directly comparable)- Long enough Samples
Scales of Scoring (approaches)
Holistic & Analytic
Scales of Scoring(where in specifications)
Criterial Levels of Performance
"""impressionistic"" refers to"
- Single Score- Overall Impression
Holistic Scoring- Advantages
Very Rapid (2min)
Holistic Scoring- disadvantages
- Not reliable- if scored 4 times = reliability improved
Seperate Score for each Aspects of a Task
Analytic Scoring- Advantages
- Subskills development in individuals considered- All Aspects of performance considered by Scorer- Number of Scores = ++Reliable- Weight of grade adapted
Analytic Scoring- Disadvantages
- Time consumming to score- Attention to different aspects distract from overall effect- Frequency of errors vs. Communication
If diagnostic information is required directly from the ratings given
(Analytic or Holistic??)
Analytic scoring is essential
Calibrate the scale to be used
1 - Collecting samples of performance2 - Expert assing sample to point(s) on relevant scale3 - Result = reference points