WK 9 Flashcards

1
Q

What are data/dimension reduction and what do they do?

A

They are mathematical and statistical procedures that reduce a large set of variables to a smaller set

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
2
Q

What is the goal in principal components analysis?

A

Goal is to explain as much of the total variance in a data set as possible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
3
Q

What are the steps in principal components analysis?

A

-starts with original data
-calculates covariances (correlations) between variables
-applies procedure called eigendecompostition to calculate a set of linear composites of the original variables

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
4
Q

What does principal components analysis do?

A

It repackages the variance from the correlation matrix into a set of components, through the process of eigendecompostion

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
5
Q

What is the first component?

A

It is the linear combination that accounts for the most possible variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
6
Q

What are the second and subsequent components?

A

Second component accounts for second largest amount of variance after the variance accounted for by the first is removed
- third accounts for third largest etc

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
7
Q

What does each component account for?

A

Each component accounts for as much remaining variance as possible

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
8
Q

If variables are closely related, what number of correlations do they have, and how do we represent them?

A

If variables are closely related, they have large correlations, then we can represent them by fewer composites

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
9
Q

If variables are not very closely related, what number of correlations do they have, and how do we represent them?

A

If variables are not very closely related, they have small correlations, then we will need more composites to adequately represent them.

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
10
Q

If variables are entirely uncorrelated, how many components do we need?

A

We will need as many components as there were variables in the original correlation matrix

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
11
Q

What is eigendecomposition?

A

It is a transformation of the correlation matrix to re-express it in terms of eigenvalues and eigenvectors

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
12
Q

How many eigenvectors and eigenvalues do you have for each component?

A

There is one eigenvector and one eigenvalue for each component

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
13
Q

What are eigenvalues?

A

Eigenvalues are a measure of the size of the variance packaged into a component

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
14
Q

What do larger eigenvalues mean?

A

They mean that the component accounts for a large proportion of the variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
15
Q

What do eigenvectors provide information on?

A

They provide information on the relationship of each variable to each component

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
16
Q

What are eigenvectors?

A

They are sets of weights (one weight per variable in original correlation matrix)
e.g., if we had 5 variables each eigenvector would contain 5 weights

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
17
Q

What will the some of the eigenvalues equal?

A

The sum of the eigenvalues will equal the number of variables in the data set

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
18
Q

What is the covariance of an item with itself?

A

The covariance of an item with itself is 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
19
Q

When you add up the covariance of items, what do you get?

A

Adding these up = total variance

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
20
Q

What does a full eigendecomposition account for?

A

It will account for all variance distributed across eigenvalues so the sum of the eigenvalues must = 1

How well did you know this?
1
Not at all
2
3
4
5
Perfectly
21
Q

We use eigenvectors to think about the nature of components. To do so, what do we do?

A

We convert eigenvectors to PCA loadings

22
Q

What does a PCA loading give?

A

A PCA loading gives the strength of the relationship between the item and the component

23
Q

What is the range of PCA loadings?

A

Range from -1 to 1

24
Q

In a PCA loading, what does a higher absolute value indicate?

A

The higher the absolute value, the stronger the relationship

25
Q

What will the sum of squared loadings for any variable on all components equal?

A

The sum of square loadings for any variable on all components will equal 1
- that is all the variance in the item is explained by the full decomposition

26
Q

Where does dimension reduction come from?

A

Comes from keeping only the largest components

27
Q

What can our decisions on how many components to keep be guided by?

A
  • set a amount of variance you wish to account for
  • scree plot
  • minimum average partial test (MAP)
  • parallel analysis
28
Q

What is the simplest method we can use to select a number of components?

A

Simply state a minimum variance we wish to account for
(We then select the number of components above this value)

29
Q

What is a scree plot based on?

A

Based on plotting the eigenvalues

30
Q

What are you looking for in a scree plot?

A

Looking for a sudden change of slope

31
Q

What is the scree plot assumed to show?

A

It is assumed to potentially reflect the point at which components become substantively unimportant
(The points should drop, as variance decreases across the components)

32
Q

In a scree plot, what is inferred as the slope flattens?

A

As the slope flattens, each subsequent component is not explaining much additional variance

33
Q

On a scree plot, what is on the x-axis?

A

The component number

34
Q

On a scree plot, what is on the y-axis?

A

The eigenvalue for each component

35
Q

How do we decide what components to keep using a scree plot?

A

Keep the components with eigenvalues above a kink in the plot

36
Q

What does the minimum average partial (MAP) test do?

A

MAP extracts components iteratively from the correlation matrix

37
Q

What is the scree plot assumed to show?

A

It is assumed to potentially reflect the point at which components become substantively unimportant
(the points should drop, as variance decreases across the components)

38
Q

What is the trend we see with MAP values?

A

At first this quantity goes down with each component extracted but then it starts to increase again

39
Q

What components does MAP keep?

A

MAP keeps the components from point at which the average squared partial correlation is at its smallest (point before there is an increase)

40
Q

How do we obtain results of the MAP test?

A

using vss() function from psych package

41
Q

What is parallel analysis?

A

Parallel analysis simulates datasets with same number of participants and variables but no correlations

42
Q

What does parallel analysis compute?

A

It computes an eigen-decompostion for the simulated datasets

43
Q

What does parallel analysis compare?

A

It compares the average eigenvalue across the simulated datasets for each component

44
Q

What happens if a real eigenvalue exceeds the corresponding average eigenvalue from the simulated datasets?

A

It is retained

45
Q

How do we conduct parallel analysis in R?

A

fa.parallel() function in psych package

46
Q

What is a limitation of scree plots?

A

Scree plots are subjective and may have multiple or no obvious kinks

47
Q

What is a limitation of parallel analysis?

A

Parallel analysis sometimes suggests too many components (over-extraction)

48
Q

What is a limitation of MAP?

A

MAP sometimes suggests too few components (under-extraction)

49
Q

What should you do if your MAP and parallel analysis disagree?

A

If the two tests disagree with each other, think about parallel analysis as an absolute maximum, and MAP a minimum value -> therefore you have a range in which optimum answer is probably within

50
Q

How are component loadings calculated and how can they be interpreted?

A

Component loadings are calculated from the values in the eigenvectors and they can be interpreted as the correlations between variables and components

51
Q

In a component loading matrix, when looking at the output what are SS loadings?

A

They are the eigenvalues

52
Q

What does a good PCA solution explain?

A

It explains the variance of the original correlation matrix in as few components as possible