week 3 Data Cleaning (text books)

Question 1

What is the basic logic used to interpret all inferential statistics?

Answer 1

The basic logic used to interpret all inferential statistics is the logic of significance, and the null & alternate hypothesis

Question 2

What do parametric statistics depend on?

Answer 2

Parametric statistics depend on normality

Question 3

What is the first step in the analysis process?

Answer 3

The first step in the analysis process is always to screen the data

Question 4

Why is data screening the first step in the analysis process?

Answer 4

• To ensure accuracy
• To check assumptions
• After the screening process a decision needs to be made as to which statistical test to use

Question 5

What are the 4 steps of data screening / data analysis?

Answer 5

Screening the data involves

1. Checking the accuracy of the data entry
2. Examining any missing data to access if there is evidence that they might be systematic (i.e. caused by something specific, as opposed to being due to random errors)
3. Looking for outliers
4. Checking parametric test assumptions (e.g. normality)

Question 6

Why is it important to inspect my data file for missing data?

Answer 6

I need to consider if there is a variable with a lot of missing data, and if so, ask why; i.e.: Is it random or is there a systematic pattern?

Question 7

How do I inspect my data file for missing data?

Answer 7

o Run “Descriptives” & find out what percentage of values is missing from each of my variables
o SPSS has a Missing Value Analysis (MVA) procedure to help find patterns in missing data

Question 8

Why is caution recommended when managing missing values in my statistical analysis?

Answer 8

I should choose carefully as it can have dramatic results
This is particularly important if I am including a list of variables & repeating the same analysis for all variables (e.g. correlations among a group of variables, t-tests for a series of dependent variables)

Question 9

What is the best way to manage missing values when I come to do my statistical analysis?

Answer 9

*Pallant strongly recommends the ‘pairwise’ exclusion of missing data
• ‘exclude cases listwise’ includes cases only if it has full data on all the variables.
• ‘exclude cases pairwise’ is better as it excludes the participant only if they are missing data required for a specific analysis. They will still be included in analysis for which they have necessary information

Question 10

What should I never do when managing missing data using SPSS?

Answer 10

*I should NEVER run the ‘replace with mean option’ as it can severely distort the results of my analysis, particularly if I have a lot of missing values
• ‘replace with mean’ calculates the mean value for the variable & give every missing case this value

Question 11

What do Tabachnick & Fidell advise when dealing with missing data?

Answer 11

The pattern of missing data is more important than the amount of data missing

Question 12

So at what percentage of missing data do we start to grow concerned?

Answer 12

o If less than 5% of data are missing in a random pattern from a large data set, the problems are less serious & almost any procedure for handling missing values yields similar results
o If, however, a lot of data are missing from a small to moderately sized data set, problems can be very serious
o Unfortunately, there are as yet no firm guidelines for how much missing data can be tolerated for a sample of a given size

Question 13

How many types of missing data are there?

Answer 13

There are 3 types of missing data:

MCAR – missing completely at random
MAR – missing at random (ignorable responses)
MNAR – missing not at random or not-ignorable

Question 14

What can you tell me about MCAR?

Answer 14

MCAR – missing completely at random
The distribution of missing data is unpredictable in MCAR
This is the best of all possible worlds if data must be missing

Question 15

What can you tell me about MAR?

Answer 15

MAR – missing at random (ignorable responses)

The pattern of missing data is predictable from other variables in the data set

Question 16

What can you tell me about MNAR?

Answer 16

MNAR – missing not at random or not-ignorable

The missingness is related to the variable itself & therefore cannot be ignored

Question 17

What do Tabachnick & Fidell advise when dealing with missing data?

Answer 17

The pattern of missing data is more important than the amount of data missing

Question 18

So at what percentage of missing data do we start to grow concerned?

Answer 18

o If less than 5% of data are missing in a random pattern from a large data set, the problems are less serious & almost any procedure for handling missing values yields similar results
o If, however, a lot of data are missing from a small to moderately sized data set, problems can be very serious
o Unfortunately, there are as yet no firm guidelines for how much missing data can be tolerated for a sample of a given size

Question 19

How many types of missing data are there?

Answer 19

There are 3 types of missing data:

MCAR – missing completely at random
MAR – missing at random (ignorable responses)
MNAR – missing not at random or not-ignorable

Question 20

What can you tell me about MCAR?

Answer 20

MCAR – missing completely at random
The distribution of missing data is unpredictable in MCAR
This is the best of all possible worlds if data must be missing

Question 21

What are outliers & how can you identify them?

Answer 21

1. Identify them via box plots or eyeballing histograms
   o If outliers are identified: Standardise them by converting the raw scores to z scores
   o Z scores in excess of +/- 3.29 (p<.001) may be regarded as outliers
   o When N is large (large sample size) a few extreme scores are expected

Question 22

Why are outliers a problem?

Answer 22

• Outliers are a particular problem for parametric tests
• Outliers are extreme scores which distort the results
• Outliers drag the mean towards themselves and exert ‘more than their fair share’ of influence over the mean & the results of statistical tests
• Outliers can be the cause of non-normality

Question 23

What is the temptation when dealing with missing data?

Answer 23

• It is tempting to assume data is missing at random, however, we can test it & we should
o It is possible to create dummy variables with 2 groups: cases with missing & cases with non-missing values & perform a test of mean differences between the 2 groups
o If there are no differences, decisions about how to handle missing data are not so critical
o If there are significant differences, & η² is substantial, care is needed to preserve the cases with missing values from other analysis

Question 24

What options does SPSS offer us to deal with Missing Data?

Answer 24

SPSS MVA (Missing Values Analysis) is specifically designed to highlight patterns of missing data as well as to replace them in a data set
The resulting Little's MCAR test tells us what to do next

Question 25

What does Little’s MCAR test establish?

Answer 25

• Little’s MCAR test establishes whether the data are missing completely at random
o A statistically nonsignificant result is desired.
o MCAR can be inferred where p = greater than .05

Question 26

What are the results we do not wish to see from a Little’s MCAR test?

Answer 26

o MAR can be inferred if the MCAR test is statistically significant but missingness is predictable from variables (other than the DV) – this is established using the ‘Separate Variances t Tests’
o MNAR is inferred if the t test shows that missingness is related to the DV

Question 27

Why are outliers a problem?

Answer 27

• Outliers are a particular problem for parametric tests
• Outliers are extreme scores which distort the results
• Outliers drag the mean towards themselves and exert ‘more than their fair share’ of influence over the mean & the results of statistical tests
• Outliers can be the cause of non-normality

Question 28

What are some of the options available to us when dealing with outliers?

Answer 28

o Delete them but this changes the nature of the sample (only delete if it’s believed the score is not really from the population of interest (e.g. below average IQ when intended population is average IQ)
o Change outliers so they are only one unit more extreme than the next most extreme score (e.g. change from 45 to 24 where next most extreme score is 23).
 This ensures outlier still exerts influence, but no more than its fair share
o Can transform the data or use nonparametric statistics (as these are based on ranks thus not affected by extreme scores, and do not require normality)

Question 29

What are some of the options available to us when dealing with outliers?

Answer 29

o Delete them but this changes the nature of the sample (only delete if it’s believed the score is not really from the population of interest (e.g. below average IQ when intended population is average IQ)
o Change outliers so they are only one unit more extreme than the next most extreme score (e.g. change from 45 to 24 where next most extreme score is 23).
o This ensures outlier still exerts influence, but no more than its fair share
o Can transform the data or use nonparametric statistics (as these are based on ranks thus not affected by extreme scores, and do not require normality)

Question 30

Once we have dealt with our outliers, what does that allow us?

Answer 30

Once outliers are dealt with the data may now meet the normality assumptions so parametric tests are possible

Question 31

If we have treated our data, what should we do when writing our results?

Answer 31

It is essential to report succinctly that any treatment of outliers in the results section

Question 32

What is the second step in the analysis process (yep, all that was just the first!)?

Answer 32

The second step in the analysis process is to decide which statistical test to use

Question 33

In what circumstances would you use non-parametric tests?

Answer 33

Use non-parametric tests when:
• The data (scores on the DV) are ordinal rather than interval or ratio
• Sample sizes are small (<10, where it can be virtually impossible to conduct accurate assumption checks
• initial data screening indicates that the parametric assumptions, particularly normality, are violated
• sample sizes are small & unequal
Non-parametric tests are an alternative to dealing with outliers as they are not distorted by them, however, non-parametrics are less accurate