What is the main purpose of Factor Analysis (FA)?
FA is a dimensionality reduction technique that identifies underlying factors that explain relationships between observed variables.
Who is credited with developing Factor Analysis?
Charles Spearman, who used FA to analyze test scores and propose the idea of general intelligence.
What does Factor Analysis help with?
- Groups related variables together.
- Reduces redundancy in data.
- Helps identify latent (hidden) factors driving the data.
What is the basic mathematical model of Factor Analysis?
x_i = a_i f + e_i
Where:
- x_i = Observed variable
- f = Latent factor
- a_i = Factor loading (strength of relationship)
- e_i = Unique, unexplained variance
What do factor loadings represent in Factor Analysis?
- They measure how much an observed variable depends on a factor.
- Values close to 1 or -1 indicate a strong relationship.
- Rotations (like Varimax) help improve interpretability.
How does Factor Analysis (FA) differ from PCA?
- FA: Assumes latent factors cause observed patterns.
- PCA: Purely mathematical, no statistical assumptions.
- FA: Focuses on interpretability of factors.
- PCA: Focuses on explaining variance in the data.
How do we decide how many factors to retain?
- Keep factors with eigenvalues > 1.
- Use a scree plot to find the “elbow point.”
- Use maximum likelihood estimation for more precise results.
Why is factor rotation used in FA?
- Improves interpretability of factor loadings.
- Helps push values closer to 0 or 1.
- Varimax = Keeps factors uncorrelated.
- Oblimin = Allows factors to be correlated.
How do we know meaning to factors?
- Look at which observed variables have high loadings on each factor.
Example: If a factor has high loadings from reading and writing skills, it could represent verbal ability.
What is the difference between EFA and CFA?
- Exploratory Factor Analysis (EFA):
- Finds hidden structure without assumptions.
- Lets the data determine the factor structure.
- Confirmatory Factor Analysis (CFA):
- Tests if data fits a predefined factor structure.
- Used for hypothesis testing.
What are some limitations of Factor Analysis?
- Assumes normal distribution of variables.
- Cannot infer causation between factors and variables.
- High multicollinearity makes factors harder to interpret.
- Small sample sizes lead to unstable results.
- Replication is needed for reliable conclusions.
What should be considered when applying FA?
- Larger samples give more stable results.
- Expert interpretation is crucial.
- Results should be validated on different datasets.
What is the key benefit of Factor Analysis?
FA reduces complexity in data by grouping related variables into hidden factors, helping to uncover underlying patterns while preserving interpretability.
