Stability of the Data-Model Fit over Increasing Levels of Factorial Invariance for Different Features of Design in Factor Analysis
Received: 18 January 2021 | Revised: 28 January 2021 | Accepted: 31 January 2021 | Online: 11 April 2021
Corresponding author: D. Almaleki
Abstract
The aim of this study is to provide an empirical evaluation of the influence of different aspects of design in the context of factor analysis in terms of model stability. The overall model stability of factor solutions was evaluated by the examination of the order for testing three levels of Measurement Invariance (MIV) starting with configural invariance (model 0). Model testing was evaluated by the Chi-square difference test (Δx2) between two groups, and Root Mean Square Error of Approximation (RMSEA), Comparative Fit Index (CFI), and Tucker-Lewis Index (TLI). Factorial invariance results revealed that the stability of the models was varying over increasing levels of measurement as a function of Variable-To-Factor (VTF) ratio, Subject-To-Variable (STV) ratio, and their interactions. There were invariant factor loadings and invariant intercepts among the groups indicating that measurement invariance was achieved. For VTF ratios 4:1, 7:1, and 10:1, the models started to show stability over the levels of measurement when the STV ratio was 4:1. Yet, the frequency of stability models over 1000 replications increased (from 77% to 91%) as the STV ratio increased. The models showed more stability at or above 32:1 STV.
Keywords:
model stability, factorial invariance, level of measurement invariance, model designDownloads
References
J. S. Tanaka, "'How Big Is Big Enough?': Sample Size and Goodness of Fit in Structural Equation Models with Latent Variables," Child Development, vol. 58, no. 1, pp. 134-146, 1987. https://doi.org/10.2307/1130296
J. J. Hox, C. J. Maas, and M. J. Brinkhuis, "The effect of estimation method and sample size in multilevel structural equation modeling," Statistica Neerlandica, vol. 64, no. 2, pp. 157-170, 2010. https://doi.org/10.1111/j.1467-9574.2009.00445.x
T. A. Holster, J. W. Lake, and W. R. Pellowe, "Measuring and predicting graded reader difficulty," Reading in a Foreign Language, vol. 29, no. 2, pp. 218-244, Oct. 2017.
S. Doi, M. Ito, Y. Takebayashi, K. Muramatsu, and M. Horikoshi, "Factorial validity and invariance of the Patient Health Questionnaire (PHQ)-9 among clinical and non-clinical populations," PLOS ONE, vol. 13, no. 7, 2018, Art. no. e0199235. https://doi.org/10.1371/journal.pone.0199235
T. A. Brown, Confirmatory Factor Analysis for Applied Research, Second Edition. New York ; London: Guilford Publications, 2015.
P. J. Ferrando and U. Lorenzo-Seva, "Assessing the Quality and Appropriateness of Factor Solutions and Factor Score Estimates in Exploratory Item Factor Analysis - Pere J. Ferrando, Urbanoa, 2018," Educational and Psychological Measurement, vol. 78, no. 5, 2018. https://doi.org/10.1177/0013164417719308
U. Lorenzo-Seva et al., "Psychometric properties and factorial analysis of invariance of the Satisfaction with Life Scale (SWLS) in cancer patients," Quality of Life Research: An International Journal of Quality of Life Aspects of Treatment, Care and Rehabilitation, vol. 28, no. 5, pp. 1255-1264, May 2019. https://doi.org/10.1007/s11136-019-02106-y
D. Almaleki, "The Precision of the Overall Data-Model Fit for Different Design Features in Confirmatory Factor Analysis," Engineering, Technology & Applied Science Research, vol. 11, no. 1, pp. 6766-6774, Feb. 2021. https://doi.org/10.48084/etasr.4025
D. Almaleki, "Empirical Evaluation of Different Features of Design in Confirmatory Factor Analysis," Ph.D. dissertation, Western Michigan University, Kalamazoo, MI, USA, 2016.
O. P. John and S. Srivastava, "The Big Five Trait taxonomy: History, measurement, and theoretical perspectives," in Handbook of personality: Theory and research, 2nd ed, New York, NY, US: Guilford Press, 1999, pp. 102-138.
B. Thompson, "Ten commandments of structural equation modeling," in Reading and understanding MORE multivariate statistics, Washington, DC, US: American Psychological Association, 2000, pp. 261-283.
K. Coughlin, "An Analysis of Factor Extraction Strategies: A Comparison of the Relative Strengths of Principal Axis, Ordinary Least Squares, and Maximum Likelihood in Research Contexts that Include both Categorical and Continuous Variables," Ph.D. dissertation, University of South Florida, Tampa, FL, USA, 2013.
A. H. Blasi and M. Alsuwaiket, "Analysis of Students' Misconducts in Higher Education using Decision Tree and ANN Algorithms," Engineering, Technology & Applied Science Research, vol. 10, no. 6, pp. 6510-6514, Dec. 2020. https://doi.org/10.48084/etasr.3927
A. J. S. Morin, N. D. Myers, and S. Lee, "Modern Factor Analytic Techniques," in Handbook of Sport Psychology, 4th ed., M. Tenenbaum and R. C. Eklund, Eds. Hoboken New Jersey, USA: Wiley, 2020. https://doi.org/10.1002/9781119568124.ch51
R. K. Henson and J. K. Roberts, "Use of Exploratory Factor Analysis in Published Research: Common Errors and Some Comment on Improved Practice," Educational and Psychological Measurement, vol. 66, no. 3, pp. 393-416, Jun. 2006. https://doi.org/10.1177/0013164405282485
A. W. Meade and D. J. Bauer, "Power and Precision in Confirmatory Factor Analytic Tests of Measurement Invariance," Structural Equation Modeling: A Multidisciplinary Journal, vol. 14, no. 4, pp. 611-635, Oct. 2007. https://doi.org/10.1080/10705510701575461
J. C. F. de Winter, D. Dodou, and P. A. Wieringa, "Exploratory Factor Analysis With Small Sample Sizes," Multivariate Behavioral Research, vol. 44, no. 2, pp. 147-181, Apr. 2009. https://doi.org/10.1080/00273170902794206
E. Guadagnoli and W. F. Velicer, "Relation of sample size to the stability of component patterns.," Psychological Bulletin, vol. 103, no. 2, pp. 265-275, 1988, https://psycnet.apa.org/doi/10.1037/0033-2909.103.2.265. https://doi.org/10.1037/0033-2909.103.2.265
R. C. MacCallum and J. T. Austin, "Applications of Structural Equation Modeling in Psychological Research," Annual Review of Psychology, vol. 51, no. 1, pp. 201-226, 2000. https://doi.org/10.1146/annurev.psych.51.1.201
K. Y. Hogarty, C. V. Hines, J. D. Kromrey, J. M. Ferron, and K. R. Mumford, "The quality of factor solutions in exploratory factor analysis: The influence of sample size, communality, and overdetermination," Educational and Psychological Measurement, vol. 65, no. 2, pp. 202-226, 2005. https://doi.org/10.1177/0013164404267287
L. R. Fabrigar, D. T. Wegener, R. C. MacCallum, and E. J. Strahan, "Evaluating the use of exploratory factor analysis in psychological research.," Psychological Methods, vol. 4, no. 3, pp. 272-299, 1999, https://psycnet.apa.org/doi/10.1037/1082-989X.4.3.272. https://doi.org/10.1037/1082-989X.4.3.272
J. C. Westland, "Lower bounds on sample size in structural equation modeling," Electronic Commerce Research and Applications, vol. 9, no. 6, pp. 476-487, 2010. https://doi.org/10.1016/j.elerap.2010.07.003
S. Zitzmann and M. Hecht, "Going Beyond Convergence in Bayesian Estimation: Why Precision Matters Too and How to Assess It," Structural Equation Modeling: A Multidisciplinary Journal, vol. 26, no. 4, pp. 646-661, Jul. 2019. https://doi.org/10.1080/10705511.2018.1545232
R. Jacobucci, A. M. Brandmaier, and R. A. Kievit, "A practical guide to variable selection in structural equation modeling by using regularized multiple-indicators, multiple-causes models," Advances in Methods and Practices in Psychological Science, vol. 2, no. 1, pp. 55-76, 2019. https://doi.org/10.1177/2515245919826527
A. B. Costello and J. Osborne, "Best practices in exploratory factor analysis: Four recommendations for getting the most from your analysis," Practical Assessment, Research, and Evaluation, vol. 10, 2005, Art. no. 7.
G. P. Brooks and G. A. Johanson, "TAP: Test Analysis Program," Applied Psychological Measurement, vol. 27, no. 4, pp. 303-304, Jul. 2003. https://doi.org/10.1177/0146621603027004007
X. An and Y.-F. Yung, "Item Response Theory: What It Is and How You Can Use the IRT Procedure to Apply It." SAS Institute Inc, 2014.
B. S. Everitt, "Multivariate analysis: The need for data, and other problems," The British Journal of Psychiatry, vol. 126, no. 3, pp. 237-240, 1975. https://doi.org/10.1192/bjp.126.3.237
P. M. Bentler and D. G. Bonett, "Significance tests and goodness of fit in the analysis of covariance structures," Psychological Bulletin, vol. 88, no. 3, pp. 588-606, 1980. https://doi.org/10.1037/0033-2909.88.3.588
G. D. Garson, "StatNotes: Topics in Multivariate Analysis," North Carolina State University. https://faculty.chass.ncsu.edu/garson/PA765/statnote.htm (accessed Feb. 02, 2021).
D. L. Bandalos and P. Gagné, "Simulation methods in structural equation modeling," in Handbook of structural equation modeling, New York, NY, US: The Guilford Press, 2012, pp. 92-108.
H. W. Marsh, K.-T. Hau, and D. Grayson, "Goodness of Fit in Structural Equation Models," in Contemporary psychometrics: A festschrift for Roderick P. McDonald, Mahwah, NJ, US: Lawrence Erlbaum Associates Publishers, 2005, pp. 275-340.
R. O. Mueller and G. R. Hancock, "Structural equation modeling," in The reviewer's guide to quantitative methods in the social sciences, New York, NY, USA: Routledge, 2018, pp. 445-456. https://doi.org/10.4324/9781315755649-33
N. O'Rourke and L. Hatcher, A Step-by-Step Approach to Using SAS for Factor Analysis and Structural Equation Modeling, 2nd ed. Cary, NC, USA: SAS Institute, 2014.
F. B. Bryant and P. R. Yarnold, "Principal-components analysis and exploratory and confirmatory factor analysis," in Reading and understanding multivariate statistics, Washington, DC, USA: American Psychological Association, 1995, pp. 99-136.
I. P. Cobham, "The effects of subject-to-variable ratio, measurement scale, and number of factors on the stability of the factor model," Ph.D. dissertation, ProQuest Information & Learning, USA, 1999.
Downloads
How to Cite
License
Copyright (c) 2021 Author
This work is licensed under a Creative Commons Attribution 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain the copyright and grant the journal the right of first publication with the work simultaneously licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) after its publication in ETASR with an acknowledgement of its initial publication in this journal.
