TMT Flashcards Preview

Z3 - Testing > TMT > Flashcards

Flashcards in TMT Deck (58):
1

Dictation techniques (Characteristics)

- Traditionally (spelling measurement)
- Now (listening and writing ability measurement)
- Direct measurement of language ability
- Beginner = Fill in the blanks dictation

2

Why is it important to specify texts with many details?

- Representativeness (texts candidates should be able to read)
- Authenticity (texts nativelike-or-not in test depending on text items intended to measure)

3

Non-Manifestation of Receptive Skills (problems)

- Reading and Listening (skills involved uncertain)
- Difficult to know if items succeeded measuring skills

4

Receptive Skills (Problems)

- Exercise of Recept. Sk. doesn't mannifest itself in overt behaviour
- Challenge to make tests that demand the use of skills and result in behaviour that demonstratre Successful use of Sk.

5

What are 3 Direct Test Techniques?

1 - Oral interview (everything)
2 - Composition Writing (writing, voc, gram. etc)
3 - Summarizing - valid and direct (oral + reading comp, writing)

6

How to Improve Positive Washback

- abilities wanted to develop
- widely and unpredicably sample
- criterionreferenced/ direct
- measure course objectives
- Provide Support for Teacher
- Weigh impact against practicability

7

7 Specification of Test developpement when stating the Testing Problem

- Type
- Purpose
- Test-Takers
- Context
- Abilities to be tested (part of purpose)
- Test score consequences
- Limitations (of context)

8

Validity of a test determined

in relation to the purpose of the test
- What the test should measure)

9

What are Test Specifications?

- Blueprint of a test
- Details and information that enable to Develop a test
- Pre-test development Stage

10

Who needs test specification?

- Teachers and tests developers
- Items Writers
- Test Validators
- Test Users (e.g. Schools)

11

What is Interactiveness?

- Quality of the test itself
- Task interactive if it corresponds to test takers language ability
- If task not interactive = Not Valid

12

What is the purpose of Construct Definition in Test Specification?

- Specify Abilities to be Assessed
- Determines What we have to ask learners to do

13

Construct Definition (Example)
For a test of reading

The macro-construct is reading.
- Reading Comprehension
-- Skim
-- Scan 
-- Speed

14

Most Information Content in test specs?

+ information on content = 
- arbitrary decisions in what to include in any versions

15

What Are the 7 Test specifications

- Content
- Structure
- Timing
- Medium Channel
- Techniques
- Criterial Levels of Performance
- Scoring Procedures

16

What is Reliability?

- Quality of test score
- Consistency of measurement across different characteristics of testing situations

17

What are the 7 Reliability Factors

- Make Longer Tests
- Choose Items that Dicriminate
- Provide Clear Instructions
- Write Clear Items
- Do Not give Much Choice
- Make Objective Tests
- Make Laid out and Legible Tests

18

Authenticity and Interactiveness in Direct Testing

Authenticity - Using Construct Definition
Interactiveness - Avoiding under-over repres. of construct (measure nothing more-less)

19

Discrete-Point vs. Integrative Items

DP - Items that measure One thing at a time
II - Items that measure Many things at Once

20

Items that measure Many things at Once

Integrative

21

Items that measure One thing at a time

Discrete-Point

22

Norm-Referenced vs. Criterion-Referenced

N-R - Not Meaningful - Compared to other test takers (estimate grade)
C-R - Meaningful - gives What students know in which lang. ability

23

High test Usefulness (3 Principles)

- Overall Usefulness Important (not only 1 quality)
- Evaluate Combined effect of test qualities to get overall test usefulness
- Test Usefulness and Qualities Balance must be adapted to each testing situation

24

Test Usefulness (6 qualities)

- Reliability
- Validity
- Authenticity
- Interactiveness
- Impact
- Practicality

25

What is validity ? (+ What Approach)

- Quality of Test Scores
- Test Should Measure what it's supposed to
- Ongoing Process
- Construct validity (unitary-unified concept)

26

What is Authenticity?

- Qualitity of the test itself
- How test measures abilities authentic to real-life tasks

- Validity - if authenticity mesure -or+ constructs

27

Authenticity (Validity)

- If authenticity mesures only some constructs = Construct-Under-Representation
- If authenticity mesures more than constructs = Construct-Over-Representation

28

What is Holistic Scoring

- Point of departure = Ability
- Compensatory = All elements tested in One criterial level
(1 weak ability, but 1 mastery = Score High anyway)

29

What is the most important characteristic or quality of a test?

Construct Validity
- Test what should be tested

30

Testing (Frequency, Objectives, Tasks) 

F: One time, summative
O: Vague Broad Subjects
T: Not ideal to evaluate language abilities

31

Assessment (Frequency, Objectives, Tasks)

F: Many times, Gradual, Formative
O: Focused, Brings to Adapted Activities
T: Easily Evaluate Language Abilities

32

Validity (in Direct Testing)

+ Direct = + Validity
+ Direct = + Subjective
Make test as valid possible before reliability

33

Validity's Purpose

- Specify Test Constructs
- Make sure tasks measure nothing more, nothing less

34

Reliability (in Direct testing)

Too direct = (- Reliability)
+ Subjective = (- Reliability)
After test valid make Reliable + Objective with VALID SCORING

35

Valid Scoring

Developing Good Scales:
- Restrict teachers+raters (not rate on own taste)
- If Scale Imposed and Many Raters

36

Valid Scoring (important)

- Measuring Direct Assessment (Writing & Oral Production)

37

How to Ensure Reliability in Testing

- Maximum number of Tasks
- Restrict the Candidates
- Give No choice of Task
- Create Scoring Scales
- Train Scorers
- Have Multiple Scoring

38

Content Specification (Elements)

- Operations/Instruction
- Input Materials
- Addressees of texts
- Lenght of test and tasks
- Dialect, accent, style & speed

39

Content Specification (Operations/Instruction)

What candidates have to be able to carry out, tasks to do

40

Content Specification (Input Materials)

- Types of texts
- Topics
- Vocabulary Range
- Structural Range

41

Content Specification (Addressees of texts)

Who candidates are expected to be able to speak to

42

Content Specification (speed)

Speed of processing

43

Multiple Choice Items (Disadvantages)

- Test Only Recognition Knowledge
- Guessing may have Considerable/Unkowable impact test score
- Restrict what can be tested
- Difficult to write successful items
- Backwash may be Harmful
- Cheating facilitated

44

Functions Test Techniques should have

- Elicit Behaviour Reliable & Valid Indicator of the Ability
- Elicit Behaviour Reliavly Scored
- Economic of Time & Effort
- Beneficial Backwash Effect

45

Cloze Procedure (Characteristics)

- Not Completely direct
- Reading Tests
- Preferred over Multiple Choice Tests
- Easy to Develop & Score

46

Too specific Content Specification (Danger, Hughes)

- May go beyond understanding of components of language ability and their relationships

47

Content Specification (Hughes, Safest Procedure)

include only elements whose contribution is well established

48

Test Techniques Definition (Hughes)

Means of eliciting behavior from candidates that will tell us about their language abilities

49

Writing and Moderating Items (Steps)

1 - Sampling
2 - Writing items
3 - Moderating items

50

Sampling (Writing and Moderating items)

- Choose content (content validity + Positive Backwash)
- Sample Widely & Unpredictably

51

Writing items (Writing and Moderating items)

Based on test specification
through eyes of test-takers to avoid misunderstanding
Consider all possible response + Provide key

52

Moderating Items (Writing and Moderating items)

Show to collegues for problems
- no modification of wrong item possible = Delete Item

53

Specification of Criterial Levels of Performance (elements)

- Accuracy
- Appropriacy
- Range
- Flexibility
- Size

54

Criterial Level of Performance (Accuracy)

Pronunciation, grammatical & lexical accuracy

55

Criterial Level of Performance (Appropriacy)

Use of language appropriate to function

56

Criterial Level of Performance (Range)

(range available) Not Searching for Words

57

Criterial Level of Performance (flexibility)

Ability to initiate conversation and adapt to new topics

58

Criterial Level of Performance (Size)

long answers, can explain, develop