HC 7 - Metabolomics Data Analysis 2: Biomarkers & Validation

Question 1

PCA scores and loadings

Answer 1

Scores: describe X
Loadings: variables important differences between samples

Question 2

DPLS scores and regression coefficients

Answer 2

PLSscores: describe X + predict Y
PLS regression coefficient: important variables for discrimination

Question 3

After a biomarker pattern is found, what is done next?

Answer 3

> Statistical validation and repeat steps from clean data to biomarker pattern
Biological and/or external validation

Question 4

Biological validation

Answer 4

New experiments
> PLS calculates scores and regression coefficients, but is the analysis correct?

Question 5

PLS model: multivariate model

Answer 5

Use coherence to make a model for discrimination of classes
> you need a PLS method which describes how the coherence between the variables is
> you want a situation with less variables than groups
> composed variables into scores

Question 6

Optimistic bias in statistic models

Answer 6

All statistic models are too optimistic because the parameters are estimated from the data used for creating the model

Question 7

Validation of model principle

Answer 7

Can the classification model preduct the health status of new individuals

Question 8

Statistical multivariate model validation approaches

Answer 8

-Average prediction error
-Statistical significance (p-value) of multivariate model

Question 9

Main points Average prediction error

Answer 9

-Confidence intervals for new samples
-Separate blinded test set
-For small number of samples: cross-validation: keep one individual away and use model to predict.

Question 10

Main points statistical significance of multivariate model

Answer 10

-For H0 distribution > is the difference larger than expected for equal groups
-Permutations: classify each individual randomly, does this situation differ from the prediction model?

Question 11

Average prediction error: test set validation

Answer 11

Use b-PLS coefficients to predict class of a testset from which the class is known. Measure the prediction error of the test set.
(e.g. the y-pred is 0.3 but it is a healthy individual who should be 1 and threshold is 0.5)

Question 12

Summarize prediction errors in a confusion table. How?

Answer 12

the columns are categorized Positive and negative as true condition (for patient and healthy for example) and the rows positive and negative for prediction.
> So the cell with negative true condition and positive prediction represent the false positives, and so on

Question 13

Sensitivity

Answer 13

True Positives / (True Positives + False Negatives)

Question 14

Specificity

Answer 14

True Negatives / (True negatives + False positives)

Question 15

Cross validation

Answer 15

For small sample size, divide samples in Xtest and Xtrain
> make an optimalized model M with Xtrain (M = bPLS coefficients)
> Use M to predict Y for Xtest
> Repeat with different Xtrain and Xtest until each sample has been in a test set
> Measure numbers of misclassifications

Question 16

When using your training set as a test set, then …

Answer 16

there is a biased prediction (too optimistic for new data)

Question 17

Statistical significance validation

Answer 17

-Make permutations of class labels (or y-values)
-Make new models between X and y that should represent situations where there is no link between X and y
-Compare original prediction error with the prediction errors of many models of permutated data
-calculate p-value

Question 18

How to calculate p-value for statistical test for validation of model

Answer 18

p < (1+number of permutation models better than original) / all permutations

Question 19

PLS-DA validation with cross model validation and permutation tests

Answer 19

> measure misclassifications cross validation
calculate p-value with permutation models

Question 20

The p-value for permutation validation is significant for metabolites which …

Answer 20

are the best indicators for the coefficients and the classes

Question 21

What is the H0 for permutations?

Answer 21

During permutations, bPLS coefficients for the meaningless permutation models are made. The H0 is made from the amounts of misclassifications

Question 22

What are the expected bPLS values for the permutation models?

Answer 22

There is no relationship between X and y, therefore there is no effect for this variable and they are expected to vary around 0

Question 23

If the variable (metabolite/biomarker) is important, then the bPLS coefficients from the model should be …. than the bPLS coefficients from permutation

Answer 23

Larger or smaller.

Question 24

Visualize a plot which shows variables raked by permutated coefficients on the x-axis and coefficients on the y-axis. Black lines are shown as a sideways parabolic around 0 and a red spikey curve inbetween with red circles where the red line crosses the black line. What could this mean?

Answer 24

-The back lines indicate the 95% confidence interval H0 from permutaiton tests (H0: PLS reg coeff = 0)
-The red line: PLS Reg Coeff from original model per variable
-Red circles: points where the original model reaches out of the 95% CI of the permutation model PLS Reg Coeff and therefore is significantly unequal to the bPLS Reg Coeffs of the permutation test. These variables are good biomarkers

Question 25

Explain the following concepts in one sentence: Cross validation, training set, test set, permutation test

Answer 25

-Cross validation: with a small number of samples -Training set: to create models (estimate loadings/coefficients) -Test set: new data to test PLS model regression coefficients by predicting the class of new samples -Permutation test: random ordering of class labels

Question 26

Do you make a new model every time when you take individuals from the data for prediction in cross validation?

Answer 26

Yes, with cross-validation you create a new model every time, your training set is also always a little different because you leave out other samples. - You take out a sample and use the rest as a training set and the one sample(s) you take out use it as a test set. Instead of cross-validation, you prefer to use a good test cohort in which validation of the model can be done for individuals other than those from the training set.

Question 27

PCA loadings (P) give information about

Answer 27

The most important variables to describe variation between samples in the data

Question 28

DPLS: bPLS coefficients give most information on

Answer 28

Important variables for discrimination

Question 29

Data interpretation, what does it mean if a certain group of metabolites is more abundant in healthy or disease condition?

Answer 29

Then they may be part of a network, or pathway > pathway analysis > network visualization > Comprehensive metabolite databases

Question 30

Lets say you have a cumulative predictive accuracy list, and the top metabolite is maltose. What happens if we remove maltose from the prediction model?

Answer 30

The predictive accuracy becomes a lot smaller

Question 31

To identify groups of metabolites which are very different between the groups and therefore predictive, what can be done with the cumulative prediction accuracy list?

Answer 31

Give metabolites color codes for different metabolite groups, like glucose metabolism for maltose

Question 32

How do you recognize a specific over-represented group of metabolites in the cumulative accuracy list?

Answer 32

Lots of metabolites of same group in the top

Question 33

For biological interpretation of the group of biomarkers, it is important to change interpretation from ... to ...

Answer 33

single metabolites to groups or pathways like biochemical pathways

Question 34

Over-representation analysis input and output

Answer 34

Use a list of significant metabolites (not ordered) > metabolite set enrichment > over-representation analysis > result: biological processes which are different between the groups

Question 35

Metabolite set enrichment

Answer 35

Give weights to metabolites based on relevance (fold change or importance)

Question 36

Single Sample Profiling

Answer 36

-Metabolite concentrations fall in *normal* range, if not, compare deviation pattern to *known* causes

Question 37

Over-representation analysis uses a .... distribution

Answer 37

hypergeometric

Question 38

Binomial coefficient

Answer 38

n over k > in how many ways can I select k objects from a group of n? (n k) = n! / k!(n-k)!

Question 39

Over-representation analysis using hypergeometric distribution. Use 52 metabolites divided into 4 pathway groups (A-D) of each 13 metabolites. If you select 12 metabolites, what is the probability that 10 are from pathway A (because this was actually measured)

Answer 39

Probability using binomial coefficients = ( (13|10) * (39|2) ) / (52|12) -(13|10): 10 metabolites from 13 possible in A -(39|2): 2 metabolites from 39 of B-D. -(52|12): all possibilities of selecting 12 from 52.

Question 40

If the probability from the over-representation is very small, then ...

Answer 40

there is a systematic effect between two groups, which is important (we say the chance is too low to consider the result being chance, the disease (or class) is related to this metabolite group > pathway A differs between the two groups)