2 psychometrics and confirmatory factor analysis Flashcards

Question

What is a managed mode approach to testing?

Answer 1

secure testing centre where supervision is available controlled environment and uniformity in conditions

Answer 2

- scope - reliability - validity - acceptability - practicality - fairness - utility

Answer 3

comprehensive statistical approach that allows researchers to test complex relationships among observed and latent variables. Latent variables are not directly observed but are inferred from observed variables evaluate model fit to data handling complex relationships

Answer 4

examining the nature of latent constructs (attitudes, traits, intelligence, clinical disorders) in contrast to exploratory factor analysis, CFA explicitly tests a priori hypotheses about relation between observed variables and latent variables often the analytic tool of choice for developing and refining measurement instruments, assessing construct validity, identifying method effects, and evaluating factor invariance across time and groups part of family: structural equation modeling (SEM) 1. evaluate measurement model 2. assess structural model

Answer 5

- allows specification of highly complex hypotheses - justification of models to do so number of models positet a priori - equivalent models, justify clearly, can all models be estimated identify competing theoretical models - credibility of observed measures ensuring adequate sampling procedures, using power analysis to obtain estimates of adequate sample size justify population was SEM used for subsequent validation?

Answer 6

- data integrity - distributional assumptions of the estimation method - maximum likelihood (ML), assumption of multivariate normality (MVN) - failure to meet the assumption of MVN can lead to overestimation of chi-square statistic and inflated type 1 error and downward biased SE - anaylsis and treatment of missing data (listwise deletion, pairwise deletion, mean substiution, multiple imputation, expectation maximisation) most commonly: listwise deletion or available case analysis

Answer 7

Multivariate normality assumes that the set of variables being analyzed jointly follows a multivariate normal distribution. In simpler terms, not only should each variable be normally distributed, but their combined distribution should also form a specific shape in multidimensional space that corresponds to the multivariate normal distribution.

Answer 8

This test determines whether a sample data set matches a population with a specific distribution and if two categorical variables are independent in influencing a test statistic goodness-of-fit test test for independence - The chi-square test effectively compares the model you have defined against the ‘ideal’ model for the data. As a result, you want the chi-square test to be non-significant (p>.05) as this will allow us to infer that our predefined model is not significantly different from the ideal model of our data. - Note: as the amount of data increase, chi-square is increasingly more likely to be significant and, therefore, any interpretation of the model must be based on more than the chi-square result, especially when the dataset is large.

Answer 9

Type 1 Error (False Positive): This error occurs when the null hypothesis is true, but we incorrectly reject it. In other words, it's concluding that there is an effect or a difference when there isn't one. The probability of committing a Type 1 error is denoted by alpha (α), also known as the significance level of the test. If you set your α at 0.05, there's a 5% risk of committing a Type 1 error. An example of a Type 1 error would be a medical test incorrectly indicating that a patient has a condition when they do not. Type 2 Error (False Negative): This error occurs when the null hypothesis is false, but we fail to reject it. It means we're concluding there is no effect or difference when, in fact, there is one. The probability of committing a Type 2 error is denoted by beta (β). The power of a test, which is 1 - β, represents the test's ability to detect an effect or difference when it truly exists. An example of a Type 2 error is a medical test failing to detect a condition that is present in a patient.

Answer 10

choice of input matrix and estimation method default choice: variance-covariance matrix and ML estimation other aspects of modeling process: latent variable scales were fixed? and type of software used

Answer 11

- model fit? chi-square goodness-of-fit test; goodness-of-fit index; adjusted goodness-of-fit index, comparative fit index, root-mean-square error of approximation recommendations for cut-off values have changed: .90; .95; .97 - see below for cut-off value recommendations - is model modification practiced?

Answer 12

- parameter estimated, including variances of exogenous variables be reported (standard error) indication of variance accounted for in endogenous variables report structure and pattern coefficients - specify model and justify choice

Answer 13

- 194 studies published in 24 journals - 1409 models - theoretical formulation and data collection The majority of studies reviewed (75.5%) focused on validating or testing the factor structure of an instrument. Of these, most (77.7%) used an existing instrument, with the remainder (22.3%) reporting on validation of a new measure, examine constructs or theories (15.8%) or to assess a measurement model prior to conducting a SEM (8.8%). - 63.8% posited more than one a priori model - approximately one-fifth (21.6%) of the studies clearly indicated that measured variables had been examined for univariate normality. In only 13.4% of the articles was it clear that MVN had been examined, and in even fewer instances (3.6%) was it mentioned that data had been screened for multivariate outliers - 65% did not report missing data information - maximum likelihood was more common, 42% was not reported in 33% - 90% reported chi-square values CFI (78.4%), RMSEA (64.9%), and TLI (46.4%) - half of the studies (57.2%) stated an explicit cutoff criteria and approximately one-third (36.1%) provided a rationale for their choice of fit measures ⇒ a variety of reporting problems ⇒ no specification of matrix used, 50% did not report factor loadings or latent variable correlations, seldom mentioned whether they examined their data for normality

Answer 14

< .05 indicates a model close to ideal < .08 suggests an acceptable model > .08 is mediocre and becomes poor as the value moves further from .08.

Answer 15

- CFI compares the defined model against the baseline (i.e. the least structure, worst defined model). As a result, you ideally want the CFI to be as large as possible. As a general guideline: - >.95 is good - > .90 is acceptable - < .90 is mediocre and becomes poor as the value moves further from .90.

Answer 16

1. name your factors 2. move the datasets to the factors 3. additional output 1. additional fit measures (RMSEA, SRMR, CFI) 2. click on plots - model plots 4. chi-square test → non-significant 5. CFI → above .95 6. RMSEA and SRMR → below .08 7. factor loadings -> the larger the better