09 - Statistics and Reproducibility

Question 1

Scientific studies and statistics

Answer 1

• correlation = statcl relationship, easy to prove
• causation = implies existing mechsm between A & B, very diff to prove
• observational study = large datasets, random pop, not necessarily hyp-driven, rarely causation
• controlled study = most studies, gps vs control, can lead to causation

Question 2

Comparing groups

Answer 2

- interclass correlation coeff
ICC = var2(among gps)/(var2(among) + var2(w/i groups))
- Student's T-distribution = Gaussian w/ estimated mean
T-test assume unbiased obs° + normal distrib
- too much variables increase the number of significant ones

Question 3

Predicting and validating

Answer 3

• mapping inputs and outputs
• /!\ overfitting
• dataset divided in training + validation + testing
• qttv assessment
• – parameter sweep (make a param vary in reasonable bounds)
• – sensitivity S = ∆metric/∆param

Question 4

Reproducibility

Answer 4

• msrmt: change of results?

- analysis: another person another place gets the same results

Question 5

Unit testing

Answer 5

• indvdl units of source code are tested to assess if they are fit for use
• happy path/provoke
• test-driven programming (written before the code) or continuous integration (run everytime something has changed)

Question 6

Presenting the results

Answer 6

• visualization is important

- grammar and graphics