W4 - Testing Methods Flashcards
(75 cards)
1
Q
What 2 ways does block design consist of and test?
A
- Qualitative: ways to solve the puzzle, order of puzzle, any direction) AND
- Quantitative (time it takes) - goes up in pattern difficulty and get longer to solve it
- Tests planning, spatial awareness, motor skills, hand eye coordination, pattern recognition, construction
Interference, category information to organisation information
2
Q
What are some ways of testing the limits?
A
- Describe the pattern verbally to them OR
- write it down to test whether its a problem with their arithmetic or memory
- Ask them to draw the pattern on paper
- Show them a grid pattern to fit each block in (to test their planning)
Do it part by part - break down task into components (to test global/visuospatial skills)
3
Q
What would be a reliable test?
A
good regularity of which the test generates the same score under similar retest conditions, or different parts of the test produce similar scores
3
Q
Why do tests need to be conducted exactly word for word?
A
- the speed of tests can affect ability/memory
- non standardised instructions produce data that can’t be compared with normative data
4
Q
What are the pros and cons of using sensitivity measures in tests and diagnosis?
A
- Tests with high sensitivity will effectively detect if you have the disorder, but occasionally can give false positives (Type 1 error)
- A NEGATIVE/NORMAL result on a test with high sensitivity is useful for ruling out a disorder
5
Q
What are the pros and cons of using SPECIFICITY measures in tests and diagnosis?
A
- Tests with high specificity will effectively detect if you do not have the disorder, but occasionally can give false negatives and claim you do have the disorder when you don’t (Type 2 error)
A POSITIVE/ABNORMAL result on a test with high specificity is useful for CONFIRMING a disorder
6
Q
High sensitivity measures can sometimes produce false positives or …
A
type 1 errors
7
Q
High specificity measures can sometimes produce false negatives or …
A
type 2 errors
8
Q
What is an case study of the importance of WORK history in diagnosis?
A
- The 45-year-old longshoreman, admitted for seizures, and found a malformation AVM in the brain growing for years.
- Finding out that he had been taking jobs of decreasing mental capacity
- his lowering occupational level over time seemed to correlate with the growth of the AVM
9
Q
What is an case study of the importance of EDUCATIONAL history in diagnosis?
A
- The 52-year old real estate developer had severe multi-infarct dementia who had done 2 years of business administration.
- Produced average verbal test scores (9) erratically, getting hard questions, not easy ones, but estimated to have a high premorbid ability level of about 14.
- Thus, based on his prior educational history, it’s unlikely that he should be in the average range.
10
Q
What is the key difference between psychological testing and psychological assessment?
A
- Psych testing involves measuring someone’s ability to obtain a specific score, eg., IQ
- Psych assessment involves a variety of test scores and methods, in the context of patient history, referral, observed behaviour, to understand patient, to produce a comprehensive report
* focus on identifying and measuring cognitive and behavioural deficits as well as preserved/intact functions, patient competencies and strengths
11
Q
Why is it bad to combine similar tests together?
A
- The Digit Forward test measures attention in reverberatory store
- The Digits Backward measures memory recall
- Combining the two tests averages out and cancels out the different cognitive functions, dilutes the preciseness of a patient’s attention and memory recall abilities
12
Q
What is missed in producing a single score of cognition?
A
- Summary scores from averaging individual test scores from a battery may be within a normal range - but deviations between tests can be significant and missed, leading to misclassification or undermining a potential deficit
13
Q
What is the Rationale of Deficit Measurement - Premorbid Function?
A
- Claims that deficits can be identified by finding the patients premorbid level of cog. functioning from old historical data and comparing to current scores
- evaluated in relation to recent history, patient behaviour, any educational/cultural deprivation, and the examiner’s knowledge of neuropsych impairment patterns intra-individual comparisons of abilities, skills and behaviours
14
Q
Which is more commonly used, the direct or indirect measurement of deficit?
A
Indirect Measurement of Deficit
15
Q
What concept consists of individual comparison standards used to compare current performance against premorbid ability?
eg., cognitive tests taken from the army or schools
A
direct Measurement of Deficit
16
Q
What concept considers of INFERRING individual comparison standards to compare to current performance?
A
Indirect Measurement of Deficit (common)
Must find meaningful/defensible estimates of premorbid ability as an adequate comparison standard, risk of underestimating/overestimating their brain injury/disease
17
Q
How is the Indirect Measurement of Deficit evaluated?
A
By testing cognitive functions least likely to be impacted
18
Q
What tests are used in the Indirect Measurement of Deficit?
A
- Patient background/history
- Current Word Reading Ability (NART, 2nd Edition)
- Long Cognitive Hold Test
- Wechsler Abbreviated Scale of Intelligence
19
Q
What does the NART consist of?
(As a test of reliable estimate premorbid ability)
A
- 50 phonetically irregular words on cue cards, may have different pronunciations, eg., ‘aisle’ or ‘cellist’ or ‘bouquet’
- Reliable as a pre-morbid indicator of ability as vocabulary correlates best with generalising overall cognitive ability level
- Helpful as vocabulary is normally unaffected by most nonaphasic brain disorders
- Age/sex = test covers the 20-70 years, can be used with up to 84 years, very little differences in sex
20
Q
What are 5 limitations of the National Adult Reading Test (NART)?
A
- Suitable for patients with IQ between 69-131, but less helpful for low IQ participants
- Language = test is not helpful if patient’s first language is NOT English
- Occupation = not helpful for patients with speaking skills in their job as their high vocabulary might overestimate their premorbid ability
- Social Class = less suitable for those without normal reading abilities, might underestimate premorbid ability
- Lower generalisability = relationship between NART and memory, learning & fluency is less correlated compared to premorbid IQ scores
21
Q
What is involved in the Long Cognitive Hold Test?
A
- Vocabulary = hear a word and define the word, increasing difficulty and lots of trials
- good estimate of premorbid ability, as vocabulary usually remains intact in deteriorating patients, even after memory and reasoning worsen
- correlates highly with education
22
Q
How can the Long Cognitive Hold Test help predict dementia and depression?
A
- If patients vocabulary score is equal or greater than 2x the block design score, patient is most likely to have dementia,
- 74% accuracy rate to predict dementia and depression
- 2 groups of dementia and depression tested,
- block design was a lot worse in dementia while vocabulary intact between both groups in
23
Q
What test has replaced Picture Completion?
A
Matrix Reasoning
24
What are some criticisms of Long Cognitive Hold Test Vocabulary?
1. Patients with LH lesions often suffer from verbal aphasia, have to do lexical orthographic tests that don’t require them to speak
2. Vocab with its spoken components is more vulnerable to brain damage than verbal/written tests which only need 1-2 word answers
25
What is Krill's Picture Completion task for?
1. Picture Completion task asks participants to identify what is missing in a picture to add PERCEPTION REASONING function to estimate premorbid function
2. (taking vocab + Picture completion + demographic data = estimate)
26
What does the Matrix reasoning task consist of?
Choose one of the picture options to fill in the matrix
The correct answer lies aligns with the picture alongside it (left to right) or below/above it, NOT diagonally see schematic
replaced picture completion
27
What are the two exceptions to using the Vocab/Picture test?
1. If there is evidence that their premorbid ability is cognitively higher than the 2 tests predict from the WAIS score, i.e., from grades, life history, WAIS scores
2. If the high scores come from Digits Forward or Letter-Number sequencing or other WAIS versions
28
What does the similarities test do?
1. involves ask a participant how 2 words are alike, eg,. banana/apple, two and seven, enemy and friend
29
What 4 tests of premorbid ability does the Wechsler Abbreviated Scale of Intelligence (WAIS) use?
1. Block design
2. Similarities
3. Matrix reasoning
4. Vocabulary test
30
What does the Demographic Variables Formula calculate?
1. occupation/education + test scores (either highest WAIS scores or original ‘hold’ tests)
31
What should you use as premorbid ability if all of the test scores are similar?
take an average/IQ, or if one is higher you can look at the higher one - looking for their best ability
32
What are the limitations to using the NART or WAIS as pre-morbid ability?
1. None of these methods satisfy the clinical need for making a sufficient estimate of premorbid ability, but they are still important to implement/consider
33
How should high WAIS scores be interpreted?
1. High WAIS scores can estimate PMA but need to consider the VARIABILITY across subtests
2. Comparing scaled scores to subtest pairs
3. Comparing single subtest scaled scores to averaged scale scores on groups of subtests
4. Comparing intersubtest scatter between highest/lowest subtest scaled scores
34
What should you do and what should you avoid in tests for pre-morbid ability?
DO
1. Use the most primary score, eg., Digits Span Forward/Backward
AVOID -
1. Do not use test batteries
2. Do not use overall test battery scores - WAIS-V averaged scores
3. Do not use scale scores, eg., memory
4. Do not use averaged test scores of Digit Span
5. Do not combine scores from tests that measure different cognitions - speed & memory
35
What 4 things does the best performance method use to establish premorbid ability?
THPP
1. test scores
2. historical data
3. population norms
4. premorbid test data
35
Why is the best performance method useful, what does it use?
1. Uses evidence of pre test scores or academic/work achievement (in scores or behaviour)
2. Once the highest score/level of function has been identified, it is compared with all other aspects of current patient functioning
3. Helpful to infer premorbid status in patients with undistinguished school or vocational careers
36
What does the Best Performance method assume about PERFORMANCE AND COGNITION?
1. Performance levels in healthy people on most cognitive tests provide a reasonable estimate of performance level on other cognitive tasks, allows for estimate of premorbid ability level
2. Cognitive performance can be heightened or reduced based on external factors, e.g., brain injury, cultural deprivation, anxiety, poor work habits but it is NOT possible to perform higher than one's developmental limits permit
36
What does the Best Performance method assume about intact and impaired ability in BD patients?
that the least depressed abilities of BD patients may serve as the strongest indicators of premorbid cognitive functioning
37
What does the Best Performance method assume about BIOLOGICAL LIMITS?
1. Overachievers do not transcend their biological/developmental limits - pouring a lot of cognitive resources into mastering 1-2 skills and usually at the expense of developing others
2. Few people consistently perform at their maximum level of functioning, as functioning can be affected by educational deficiencies, impulsivity, test anxiety or simply disinterest
* Performance of any task may be the best that can be done at that time but still only indicates a floor, not ceiling, level of abilities of that task
38
What does the Best Performance method assume about CHANCE VARIATIONS?
1. Under the limits of chance variations, the ability to perform a task is at least as high as the person’s highest performance level of that task - and cannot be any lower
* Counterintuitive = as bad performance with a couple of superior indicators might immediately seem indicative of superior premorbid ability
2. Must remember that the prerequisite to skills/knowledge is the ability to learn it to begin with, eg., someone who gets capitals wrong but famous people right might have better premorbid abilities than it seems
39
What does the Best Performance method assume about USING DIFFERENT SOURCES OF PATIENT HISTORY?
1. Premorbid ability can be inferred from interview observation, family/friends information, school and work grades, army rank, academic qualifications, creations and writing
2. superior academic achievement can infer superior premorbid ability regardless of current mental state
3. Should look at as many sources as possible to avoid underestimating patient’s premorbid ability
* Examples, motor paralysis can conceal previous fine motor skills
* shyness/anxiety can conceal advanced vocabulary
40
What are 5 advantages of BPM?
1. Takes into account a broad range of the patient’s abilities
2. Good estimations of premorbid ability can act as a comparison standard for evaluating deficit
3. Comprehensive scope of one’s cognitive functions reduces clinician’s bias of certain patient populations
4. Not bound to one battery of tests, e.g., can utilise children’s test for patients with very low functioning to help uncover any residual cognitive abilities
5. Can infer intelligence from several high scores instead of just one
41
What are the 3 main criticisms of the best performance method?
1. Should never rely on a single test score as an indicator of premorbid ability
* Previous BPM based off the high scores of an intelligence battery (WIS-A), but this is always higher than a general summation score/IQ score - as IQ scores are a mean of both low and high scores, so high WIS-A scores are not a good predictor of the WIS-A IQ
2. Overachievers with superior general knowledge, arithmetic or vocabulary scores may be inflated from parental or academic pressure instead of reflecting innate ability
3. Memory scores are the least reliable indicators of premorbid ability/intelligence
42
What types of tests are the least reliable indicators of premorbid ability/intelligence?
memory scores
43
What do marked discrepancies in test performance indicate?
1. Indication of disease, developmental abnormalities, cultural deprivation, emotional disturbances - things that may interfere with their full potential
44
What's a limitation of concluding that a marked discrepancies indicates an underlying disease?
Large discrepancies do occur in healthy controls,
needs extra information to determine whether performance reflects actual impairment or is within normal variation of cognitive functioning
45
What is the range of scatter discrepancies?
Scatter discrepancies of different cognitive performance are in the 0–4 point range
46
If patients with originally HIGH scores who now demonstrate LOW performance levels, then cognitive deficit can be ....
strongly inferred with confidence
46
Deficit Measurement Paradigm: After premorbid estimate has been established, how does the examiner evaluate any presence of cognitive deficit?
1. Comparing the patient’s present cognitive performances against expected/comparison standards (expected vs. observed scores)
2. tested for statistical significance - to determine whether the present scores indicate any cognitive deficits
3. Comparisons are done for each domain of test scores, or compared with estimates of original ability when premorbid data is not available
4. Significant discrepancies between multiple test scores indicate a pattern of deficit and can aid in diagnosis, when compared to patterns of deficit in other neurological disorders, eg., alzheimer's, multiple sclerosis
47
Patients with originally SUPERIOR scores who now demonstrate AVERAGE performance level can
... only indicate deficits
48
Patients with originally HIGH scores now demonstrates AVERAGE performance levels, cognitive deficit ....
cannot be inferred even though performance is lower than expected, and these low scores need to be evaluated in combination with other types of info
May be due due to normal levels of score fluctuation
49
What is neuroinformatics and how can it help in diagnosis?
1. Normative data and deficit performance data from patient groups with specific diseases both provide comparison information to a patient's test scores
Neuroinformatics with growing amount of normative, genetic, clinical and historical information data also helps diagnosis
50
What are the 3 main things that the deficit measurement method can do?
1. helps identify patterns of cognitive impairment in psychiatrically, developmentally disabled or culturally/educationally deprived people
2. Aid in diagnosing a neurological disorder
3. Estimate premorbid functioning of children - more complicated as brain injury is interacting with normal stages of development
51
When was the WAIS scale established and how many editions?
1. used since 1955 - very reliable/valid technique
2. improved with 5 editions, broke the brain down into specific components of intelligence in the fourth edition in 2008, currently WAIS-V from 2024
52
What are the 5 retained 5 subsets of intelligence in the WAIS?
1. verbal comprehension
2. visuospatial
3. processing speed
4. fluid reasoning
5. working memory
53
Order of tests -
1. When is it better to start with easy tests?
1. When is it better to start with harder tests?
1. To ensure patient feels comfortable with the exam, keeps self-esteem/self-efficacy/motivation
2. more difficult tests can be given towards the beginning of the examination when the patient is less fatigued
54
Does test order seem to impact performance?
Are there any exceptions to this?
1. Overall, order of tests does not significantly impact a patient's performance, but may make some examiners uncomfortable.
2. Exceptions include: Finger Tapping or Digit Span Coding is generally slower when tested later in the examination, arguably due to fatigue
3. No important effects appeared when both WAIS-III and the Wechsler Memory Scale-III (WMS-III) batteries were given in different order
55
When is testing the limits usually applied?
When is TTL done?
1. when the examiner suspects the patient’s impaired performance is being influenced by some other function that is different to the one being tested
2. Testing the limits does not affect test scoring or standard procedures and is only done after the test is completed
55
What 3 things are important to remember regarding the Interval proceeding delayed trials?
1. Important to keep the patient ENGAGED during interval proceeding delayed trials on learning tests with another task
2. Need to consider the interval time and modality: eg., doing a verbal fluency test in the middle of a word list test might impact performance of the original test
3. The choice of interval tasks is dependent on whether the examiner wants to produce high or low levels of interference / if interference susceptibility is being measured
56
If a patient who performs arithmetic at an average level - and then allowed to write down the elements of the problem and suddenly performs the task at a superior level -
they have intact ----
and impaired ----
1. Intact arithmetic ability
2. Impaired auditory span and mental tracking
57
Are BD patients susceptible to practice effects in repeated tests?
Both BD and healthy patients are susceptible to practice effects with the same tests, this has also shown up in PET scans with shifts in brain activation patterns
58
What types of tests are more liable to show MORE SIGNIFICANT practice effects?
Tests with a
1. large speed component
2. unfamiliar/infrequent responses
3. single solutions
4. memory tests (BD patients without memory issues often learn the material)
59
What's the biggest limitations in repeated testing?
1. The lack of available alternative and well standardised tests / parallel forms, especially memory tests
2. General test taking over repeated examinations may also enhance performance, regardless of the test taken
3. Patients are often less anxious and more familiar with the examiner and procedure, allowing them to develop test-taking strategies / test wiseness
60
When do the greatest practice effects occur, and how many years need to pass before the advantage is gone?
1. Greatest practice effects off between 1st and 2nd sessions
2. 7-13 years need to pass before test taking advantage is gone in patients
61
How do you attenuate practice effects?
1. Find alternate / parallel forms
2. Do 2+ baseline examinations before doing the experimental condition
61
What are the 3 limitations of parallel forms
1. Limited availability of parallel forms
2. Limited interform reliability
3. May NOT have equal level of task difficulty, leading to unwanted variance in the data rather than practice effects
62
Are practice effects a big concern across age?
practice effects are quite minimal, depending on age/condition, most in digit span and Trail Making, but decreases with age
No clear pattern of age influencing practice effects, WIS-A practice effects may be slightly greater in younger than older individuals
Practice effects still occurring for older adults 65-79 years of age, but not 80+ years
63
WIS-A: What scales are least stable over time?
retention scores
63
Is the WIS-A robust?
What scales are more stable over time?
Yes
Verbal knowledge and skills
64
Can practice effects be clinical indicators?
Yes - When individuals do not show any practice effects on memory and category fluency, it can be an indicator of mild cognitive impairment in older adults
Those with a temporal lobectomy who do not improve on retests over time might reflect a learning deficit from their preoperative condition
65
What is the difference between technicians and neuropsychologists?
1. Technicians are hired to administer and score routine tests
2. While the neuropsychologist conducts more specialised testing, reporting writing, interviewing and patient consults
However, some neuropsychologists reply completely on the scores/observations from technicians in their report writing (eg., blind analysis)
66
What are the 4 pros of using technicians?
1. Saving time, allowing NP to see more patients
2. Objective data collection in research projects (where qualitative data is irrelevant)
3. Paid less, saving costs to patients/research grants
4. Patient is benefited from being observed by two clinicians, lowering the risk of important information being missed
67
What are the 5 cons of using technicians?
1. Over relying on technicians observations and data collection for reports, leads to blind analysis of patients
2. Some NPs consider themselves like doctors, just analysing brain scores, but neuropsychological assessment requires a holistic approach
3. Some technicians do not keep a record of how the patient performs
4. Some technician’s observations are limited by insufficient training and lack of practical experience
5. Patients may have an unfulfilling experience if they are mostly seen by a technician and only briefly visited by an NP who is not interested in them as people
68
What are the main training requirements for technicians?
1. Psychometric technicians are enrolled in formal training programs and normally have bachelor degrees in psychology or related subjects
as a minimum
2. Must be limited to scoring and doing tests under NP supervision
