Cognitive Testing and Statistics

Question 1

What is cognition?

Answer 1

Cognition refers to “the mental action or process of acquiring knowledge and understanding through thought, experience, and the senses”

Including thinking, attention, language, learning, memory and perception

Question 2

What cognitive mechanisms build upon each other to create more sophisticated functions?

Answer 2

Attention & information processing speed –>
Working memory –>
Executive functions & memory

Question 3

What do the trail making and letter fluency cognitive tasks measure?

Answer 3

Trail making = measures processing speed and aspects of executive function

Letter fluency = measures semantic abilities, primarily word generation

Question 4

Explain the trail making task

Answer 4

Trail making A = Connect the dots in one continuous line, starting at 1, finishing at 8, going through the numbers in regular ascending order

Trail making B = Connect the dots in one continuous line, starting at 1, finishing at D, alternating from a number to a letter and back again
Don’t lift the pen from the paper

Question 5

How is the trail making task scored?

Answer 5

The A condition is used as a baseline measure of motor speed (which has a component processing speed)
We subtract the A time from the B time to get the main measure of higher cognitive functions like task switching, while controlling for motor speed
In the real test an administrator keeps track of any errors and corrects them during the test itself

Question 6

Explain the letter fluency test

Answer 6

You are going to be given a letter of the alphabet and then have a minute to write down as many words you can think of that begin with that letter

Cue = C Answers = cat, car, carry, carnivore, carnival, crispy

Question 7

What are the rules of the letter fluency task

Answer 7

Rule 1 = No proper nouns (place names, person’s names or product names

Rule 2 = no modifying of a word you’ve already said - if you said ‘carry’ you can’t say ‘carries’ or ‘carried’

Rule 3 = no gibberish

Question 8

How do you score the letter fluency task?

Answer 8

The main score used on this is the total correct words, which is a measure of semantic abilities

MoCA = a point for 11 words or more

Question 9

What is a normal distribution?

Answer 9

A normal distribution is where there is a clear trend for the value of most points to cluster around a central mean with equally increasing rarity on either side of this
Many things do not follow this pattern, though we tend to use it as a starting point when considering how to conduct our statistics

Question 10

Why is the normal distribution important in cognitive testing?

Answer 10

If we are happy that our variables form a normal distribution, there are a number of mathematical assumptions we can make about them which determine how we analyse them statistically
This is a deeply complex subject, but we can largely boil the central issue down to asking if the mean value is a fair way of representing what is average

Question 11

What are the characteristics of a parametric sample?

Answer 11

Parametric tests are those that assume that the sample data comes from a population that follows a probability distribution — the normal distribution — with a fixed set of parameters

As our sample becomes more skewed, the mean is increasingly pulled in one direction by extreme values
The median and the mode become more representative of what is most expected

Question 12

What is a positive and negative skew?

Answer 12

Positive skew = right-skewed distribution, long tail on its right side (long, tapering end of the distribution), the mean is greater than its median

Negative skew = left-skewed distribution, long tail on its left side - e.g., a histogram showing test scores with a negative skew shows majority of pps scored above average and only a small proportion of students scored very low scores, mean is always less than its median

Question 13

What is the difference in parametric and non-parametric samples when it comes to statistics?

Answer 13

Parametric statistics use mean values in their calculations
Non-parametric alternatives make additional corrections to try and account for the skew

Question 14

What is variance?

Answer 14

The variance of a sample is essentially the average difference of all individual data points from the mean
It quantifies how tightly clustered (or not) the data is around the mean
It is a slight overestimation, of the sum of squares, of the difference between each individual data point and the mean

Question 15

What is standard deviation?

Answer 15

Because the variance is a version of our data after we have squared everything, its value is greatly increased compared to our original units of measurement
The SD is just the square root of the variance, so it re-converts it back into proportion with our data

Question 16

What is the relationship between the mean and SD in a normal distribution?

Answer 16

The SD holds a consistent mathematical relationship with a normal distribution
As we travel x SD’s away from the mean, the % of data points included in our range increases by set amounts

Question 17

What probability are scientists most interested in?

Answer 17

Most interested when something has less than 5% probability of happening by chance
Often find people reporting if a single data point is more than 2 SD’s from the mean, which is essentially an attempt at saying - that there is at most 5% chance that your new data point is from the same population as the one you’ve measured

Question 18

What increases the confidence of the results?

Answer 18

More samples = more confidence

It is statistically better to have a lot of pps in our study than only a few pps in our study

If we measure the heights of 5 people, an anomalous result will have a greater proportionate effect on the mean and spread of values

The more samples we have the more confident we become that our means etc are representative of that variables true pattern

Question 19

What is a p value?

Answer 19

A number calculated from a statistical test that describes how likely you are to have found a particular set of observations if the null hypothesis were true

It is essentially the % probability that the data you have observed has happened by chance

It is convention in statistical tests that if something has a p<0.05, i.e., that there is less than 5% probability of it happening by chance alone, that it is ‘statistically significant’ - we would reject our null hypothesis

Question 20

What are two common statistical designs?

Answer 20

1. Compare a value between groups of pps or before/after an intervention = Groupwise tests
2. See how one value changes depending on another value = correlation

Question 21

What statistical test is used for parametric groupwise testing (Comparing a score between two groups e.g., cognitive score performance in dementia patients and healthy controls)?

Answer 21

= independent samples t-test

Question 22

How do we convert a t value to a p value?

Answer 22

Statisticians know the thresholds for t values at different probabilities
The last thing we need to convert a t statistic to a p value is the degrees of freedom, which is essentially our N except in a t-test we subtract 1 from it

Question 23

What statistical test is used for parametric groupwise testing (Comparing before and after scores e.g., blood pressure before and after a medication is prescribed)?

Answer 23

Paired samples t-test

Question 24

What factors make the t value bigger?

Answer 24

Greater distance between means
Smaller SDs
Larger Ns

Question 25

What statistical test is used to see if there is a relationship between two variables?

Answer 25

Pearson’s correlation

Question 26

What does pearson’s correlation calculate and what does it mean?

Answer 26

Calculates the covariance between your X variable and your Y variable

r (pearsons value) comes out between -1 and 1 where:
-1 = perfect negative correlation
0 = no relationship at all
1 = perfect positive correlation

Question 27

What happens if the data is skewed?

Answer 27

Because parametric tests rely on mean values a skewed distribution becomes a problem for their validity
We can still run the same experiments (groupwise or correlations) but we must substitute them with non-parametric test alternatives

Question 28

What is the non-parametric version of an independent t-test?

Answer 28

Mann-Whitney U

Question 29

What is the non-parametric version of a paired t-test?

Answer 29

Wilcoxon signed-rank

Question 30

What is the non-parametric version of a pearson correlation?

Answer 30

Spearman correlation

Question 31

What is the main difference with non-parametric tests?

Answer 31

They use ‘ranked’ versions of the data
We can understand how this is still representative of which samples are bigger than others, but massively reduces the disproportionate effect that extreme values have on the mean

Spearman correlation is a Pearson correlation, run on ranked versions of your variables

Question 32

What are multivariate models?

Answer 32

Often we want to look at how multiple variables affect an outcome all at once
For example, seeing if cognitive performance is affected by age, brain volume and years spent in education

A single multivariate model is used to test this = multiple regression

Question 33

What is the multiple comparisons problem?

Answer 33

The more tests we run in a study, the more likely we are to get a significant result by random chance
We need to be conscious of this in our study design, and in the way we approach our analyses - sometimes we need to correct for multiple comparisons

Question 34

How do we correct for multiple comparisons?

Answer 34

Bonferroni correction

= divide 0.05 by the number of comparisons we have run - this is our new significance threshold