Documentation

Documentation for the tool

This page describes the various fields in the flexible cutoff tool. For the derivation of the cutoffs, please see the derivation page for details.

Input fields

These fields are required to derive the correct flexible cutoffs based on your model and sample characteristics.

  • Overall indicators: Number of (manifest) indicators in your model. Simply count the number of indicators you have, irrespective of their belonging to a latent variable.
  • Overall latent variables: Number of latent variables (or constructs) that are estimated in your model.
  • Sample size: Size of your dataset.
  • Average factor loadings: The mean of your standardized factor loadings (lambda). It is recommended to gather all loadings beforehand and then estimate the mean. Round to one decimal place.
  • Non-normality: The non-normality of your covariance matrix. It is recommended to estimate a Satorra-Bentler-scaled chi-square statistic (or another variant of scaled statistics) and take the ratio of that statistic with the non-scaled chi-square statistic. If it is near 1, select ’normal‘. If it is close to 1.05 and below 1.19, select ‚moderate‘. If it exceeds 1.19, select ’severe‘. In case your software does not provide this, select ’normal‘.
  • Num. of additionally fixed params: This is the number of additionally fixed parameters (e.g., loadings, covariances/correlations, variances) compared to a CFA based on your model size. This feature simply changes the resulting degrees of freedom displayed below. It does not change the flexible cutoff values.
  • Width of confidence interval: The assumed uncertainty in the evaluation of the model fit. .05 is generally recommended. In some cases, .01 (more forgiving), .001 (most forgiving) and .1 (very conservative) may be justified.

Calculation fields

These fields are informative and illustrate important statistics of your model assuming a CFA. Please refer to the number of additional fixed parameters for adjusting the resulting degrees of freedom.

  • Model type: The type of model that the flexible cutoffs are based on. So far, CFA is supported only.
  • Nearest simulated data size: The closest sample size for which simulated sample sizes are available. This is your sample size rounded to the next 50 (e.g., 235 yields 250).
  • Moments: The number of moments in your model. This is based on the overall number of indicators (I) as defined by: I(I+1)/2.
  • Avg. indicators per latent variable: The overall number of indicators divided by the number of latent variables.
  • Avg. indicators per latent variable (rounded): The overall number of indicators divided by the number of latent variables rounded to an integer without decimal places. This field corresponds to the number of indicators in the simulated dataset.
  • Free indicators: The number of indicators not fixed to a certain value (free). Typically, this is equal to I-1 per latent variable and equal to the overall number of indicators minus the number of latent variables.
  • Free error variances of indicators: The number of free error variances per indicator. Typically, this is equal to the overall number of indicators.
  • Free endogenous errors: The number of free endogenous errors. Typically, this is equal to the number of latent variables.
  • Free covariances: The number of free covariances. In CFA, all possible covariances between latent variables are set free, typically.
  • Expected degrees of freedom: The resulting degrees of freedom based on the previous values. Typically, this is equal to moments minus the number of free parameters (sum of free indicators, free error variances of indicators, free endogenous errors, and free covariances).
  • Num. of free parameters: The number of free parameters. Typically, this is equal to the sum of free indicators, free error variances of indicators, free endogenous errors, and free covariances.
  • Resulting degrees of freedom: The resulting degrees of freedom after adjusting for additionally fixed parameters. If no additionally fixed parameters are available, this is equal to the expected degrees of freedom.

Output fields

Finally, these fields provide the flexible cutoffs for fit indicators (chi-square value) or fit indices. Full names, codes, cutoff, sensitivity, and recommended use are provided. Sensitivity indicates the highest absolute difference between different uncertainty levels (alpha). Typically, this is the difference between alpha = .001 (most forgiving) and .1 (very conservative). GoF indicates Goodness-of-fit indices (ideal value is close to 1), BoF indicates Badness-of-fit-indices (ideal value is close to 0).

Recommended and tested fit indices

  • Standardized Root Mean Square Residual: SRMR is a standard fit index, a BoF and very sensitive to misspecification in the structural model (e.g., misfit in covariances). Hence, it is consistently recommended in literature and in our research.
  • Comparative Fit Index: CFI is a standard fit index, a GoF and very sensitive to misspecification in the measurement model (e.g., misfit in factor loadings). Hence, it is consistently recommended in literature and in our research. CFI is conceptually related to the Relative Noncentrality Index (RNI).
  • Tucker-Lewis Index (Non-Normed Fit Index): TLI, or NNFI, is a standard fit index, a GoF and very sensitive to misspecification in the measurement model (e.g., misfit in factor loadings). Hence, it is consistently recommended in literature and in our research. TLI is typically lower than CFI, but correlates highly with CFI.
  • Root Mean Square Error of Approximation: RMSEA is a standard fit index, a BoF and sensitive to misspecification in the measurement model (e.g., misfit in factor loadings). Hence, it is consistently recommended in literature and in our research. It should be noted that RMSEA performed not as good as CFI and TLI in our research in detecting measurement model misspecification.

Other indices (not tested in the paper and/or recommended)

These are indices we did not investigate in the paper, for space limitations and as proposed by our reviewer team. We thereof do not recommend to use any of these indices to derive flexible cutoffs and cannot guarantee a proper behavior. They are only provided for transparency reasons. This is, of course, not a full list of all possible indices.

  • Chi-square value: Often referred to as T is the empirical difference between the model-implied and the provided covariance matrix. It is not recommended due to being very sensitive to sample size.
  • Chi-square value per degree of freedom: Chi-square/d.f. is the ratio between Chi-square value and the degrees of freedom of a given model.
  • Goodness of Fit Index: GFI is a standard fit index, a GoF and very sensitive to sample and model characteristics. Thus, it is consistently not recommended in literature.
  • Adjusted Goodness of Fit Index: AGFI is an adjusted GFI that compensates for the size of a model. However, alike GFI, AGFI is very sensitive to sample and model characteristics. Thus, it is consistently not recommended in literature.
  • Parsimonious Goodness of Fit Index: PGFI is a parsimony-adjusted GFI developed to further account for model parsimony. Alike GFI, it shares the same sensitivity to sample and model characteristics.
  • Incremental Fit Index (Bollen’s Non-normed Index Delta 2): IFI is a rather rarely used fit index, a GoF and is rather sensitive to misspecification in the measurement model (e.g., misfit in factor loadings). It is often referred to as BL89.
  • Relative Fit Index (Bollen’s Normed Index Rho 1): RFI is a rather rarely used fit index, a GoF and is rather sensitive to misspecification in the measurement model (e.g., misfit in factor loadings). It is often referred to as BL86.
  • Normed Fit Index: NFI is a rather rarely used fit index, a GoF and is rather sensitive to misspecification in the measurement model (e.g., misfit in factor loadings).
  • Parsimonious Normed Fit Index: PNFI is a parsimony-adjusted NFI developed to account for model parsimony.
  • McDonald’s Measure of Centrality (McDonald Fit Index): MC or sometimes MFI is a rather rarely used fit index, a GoF and is rather sensitive to misspecification in the measurement model (e.g., misfit in factor loadings).
  • Gamma Hat: GH or sometimes Gamma is a rather rarely used fit index, a GoF and is rather sensitive to misspecification in the measurement model (e.g., misfit in factor loadings).

Further readings

Hu, L., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. 

Satorra, A., & Bentler, P. M. (2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66(4), 507-514.