TMT Flashcards
1
Q
Dictation techniques (Characteristics)
A
- Traditionally (spelling measurement)
- Now (listening and writing ability measurement)
- Direct measurement of language ability
- Beginner = Fill in the blanks dictation
2
Q
Why is it important to specify texts with many details?
A
- Representativeness (texts candidates should be able to read)
- Authenticity (texts nativelike-or-not in test depending on text items intended to measure)
3
Q
Non-Manifestation of Receptive Skills (problems)
A
- Reading and Listening (skills involved uncertain)
- Difficult to know if items succeeded measuring skills
4
Q
Receptive Skills (Problems)
A
- Exercise of Recept. Sk. doesn’t mannifest itself in overt behaviour
- Challenge to make tests that demand the use of skills and result in behaviour that demonstratre Successful use of Sk.
5
Q
What are 3 Direct Test Techniques?
A
1 - Oral interview (everything)
2 - Composition Writing (writing, voc, gram. etc)
3 - Summarizing - valid and direct (oral + reading comp, writing)
6
Q
How to Improve Positive Washback
A
- abilities wanted to develop
- widely and unpredicably sample
- criterionreferenced/ direct
- measure course objectives
- Provide Support for Teacher
- Weigh impact against practicability
7
Q
7 Specification of Test developpement when stating the Testing Problem
A
- Type
- Purpose
- Test-Takers
- Context
- Abilities to be tested (part of purpose)
- Test score consequences
- Limitations (of context)
8
Q
Validity of a test determined
A
in relation to the purpose of the test
- What the test should measure)
9
Q
What are Test Specifications?
A
- Blueprint of a test
- Details and information that enable to Develop a test
- Pre-test development Stage
10
Q
Who needs test specification?
A
- Teachers and tests developers
- Items Writers
- Test Validators
- Test Users (e.g. Schools)
11
Q
What is Interactiveness?
A
- Quality of the test itself
- Task interactive if it corresponds to test takers language ability
- If task not interactive = Not Valid
12
Q
What is the purpose of Construct Definition in Test Specification?
A
- Specify Abilities to be Assessed
- Determines What we have to ask learners to do
13
Q
Construct Definition (Example)
For a test of reading
A
The macro-construct is reading.
- Reading Comprehension
- Skim
– Scan
– Speed
14
Q
Most Information Content in test specs?
A
+ information on content =
- arbitrary decisions in what to include in any versions
15
Q
What Are the 7 Test specifications
A
- Content
- Structure
- Timing
- Medium Channel
- Techniques
- Criterial Levels of Performance
- Scoring Procedures
16
Q
What is Reliability?
A
- Quality of test score
- Consistency of measurement across different characteristics of testing situations
17
Q
What are the 7 Reliability Factors
A
- Make Longer Tests
- Choose Items that Dicriminate
- Provide Clear Instructions
- Write Clear Items
- Do Not give Much Choice
- Make Objective Tests
- Make Laid out and Legible Tests
18
Q
Authenticity and Interactiveness in Direct Testing
A
Authenticity - Using Construct Definition
Interactiveness - Avoiding under-over repres. of construct (measure nothing more-less)
19
Q
Discrete-Point vs. Integrative Items
A
DP - Items that measure One thing at a time
II - Items that measure Many things at Once
20
Q
Items that measure Many things at Once
A
Integrative
21
Q
Items that measure One thing at a time
A
Discrete-Point
22
Q
Norm-Referenced vs. Criterion-Referenced
A
N-R - Not Meaningful - Compared to other test takers (estimate grade)
C-R - Meaningful - gives What students know in which lang. ability
23
Q
High test Usefulness (3 Principles)
A
- Overall Usefulness Important (not only 1 quality)
- Evaluate Combined effect of test qualities to get overall test usefulness
- Test Usefulness and Qualities Balance must be adapted to each testing situation
24
Q
Test Usefulness (6 qualities)
A
- Reliability
- Validity
- Authenticity
- Interactiveness
- Impact
- Practicality
25
What is validity ? (+ What Approach)
- Quality of Test Scores
- Test Should Measure what it's supposed to
- Ongoing Process
- Construct validity (unitary-unified concept)
26
What is Authenticity?
- Qualitity of the test itself
- How test measures abilities authentic to real-life tasks
- Validity - if authenticity mesure -or+ constructs
27
Authenticity (Validity)
- If authenticity mesures only some constructs = Construct-Under-Representation
- If authenticity mesures more than constructs = Construct-Over-Representation
28
What is Holistic Scoring
- Point of departure = Ability
- Compensatory = All elements tested in One criterial level
(1 weak ability, but 1 mastery = Score High anyway)
29
What is the most important characteristic or quality of a test?
Construct Validity
- Test what should be tested
30
Testing (Frequency, Objectives, Tasks)
F: One time, summative
O: Vague Broad Subjects
T: Not ideal to evaluate language abilities
31
Assessment (Frequency, Objectives, Tasks)
F: Many times, Gradual, Formative
O: Focused, Brings to Adapted Activities
T: Easily Evaluate Language Abilities
32
Validity (in Direct Testing)
+ Direct = + Validity
+ Direct = + Subjective
Make test as valid possible before reliability
33
Validity's Purpose
- Specify Test Constructs
- Make sure tasks measure nothing more, nothing less
34
Reliability (in Direct testing)
Too direct = (- Reliability)
+ Subjective = (- Reliability)
After test valid make Reliable + Objective with VALID SCORING
35
Valid Scoring
Developing Good Scales:
- Restrict teachers+raters (not rate on own taste)
- If Scale Imposed and Many Raters
36
Valid Scoring (important)
- Measuring Direct Assessment (Writing & Oral Production)
37
How to Ensure Reliability in Testing
- Maximum number of Tasks
- Restrict the Candidates
- Give No choice of Task
- Create Scoring Scales
- Train Scorers
- Have Multiple Scoring
38
Content Specification (Elements)
- Operations/Instruction
- Input Materials
- Addressees of texts
- Lenght of test and tasks
- Dialect, accent, style & speed
39
Content Specification (Operations/Instruction)
What candidates have to be able to carry out, tasks to do
40
Content Specification (Input Materials)
- Types of texts
- Topics
- Vocabulary Range
- Structural Range
41
Content Specification (Addressees of texts)
Who candidates are expected to be able to speak to
42
Content Specification (speed)
Speed of processing
43
Multiple Choice Items (Disadvantages)
- Test Only Recognition Knowledge
- Guessing may have Considerable/Unkowable impact test score
- Restrict what can be tested
- Difficult to write successful items
- Backwash may be Harmful
- Cheating facilitated
44
Functions Test Techniques should have
- Elicit Behaviour Reliable & Valid Indicator of the Ability
- Elicit Behaviour Reliavly Scored
- Economic of Time & Effort
- Beneficial Backwash Effect
45
Cloze Procedure (Characteristics)
- Not Completely direct
- Reading Tests
- Preferred over Multiple Choice Tests
- Easy to Develop & Score
46
Too specific Content Specification (Danger, Hughes)
- May go beyond understanding of components of language ability and their relationships
47
Content Specification (Hughes, Safest Procedure)
include only elements whose contribution is well established
48
Test Techniques Definition (Hughes)
Means of eliciting behavior from candidates that will tell us about their language abilities
49
Writing and Moderating Items (Steps)
1 - Sampling
2 - Writing items
3 - Moderating items
50
Sampling (Writing and Moderating items)
- Choose content (content validity + Positive Backwash)
- Sample Widely & Unpredictably
51
Writing items (Writing and Moderating items)
Based on test specification
through eyes of test-takers to avoid misunderstanding
Consider all possible response + Provide key
52
Moderating Items (Writing and Moderating items)
Show to collegues for problems
- no modification of wrong item possible = Delete Item
53
Specification of Criterial Levels of Performance (elements)
- Accuracy
- Appropriacy
- Range
- Flexibility
- Size
54
Criterial Level of Performance (Accuracy)
Pronunciation, grammatical & lexical accuracy
55
Criterial Level of Performance (Appropriacy)
Use of language appropriate to function
56
Criterial Level of Performance (Range)
(range available) Not Searching for Words
57
Criterial Level of Performance (flexibility)
Ability to initiate conversation and adapt to new topics
58
Criterial Level of Performance (Size)
long answers, can explain, develop
