Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research: A Monte Carlo Approach

Yo In'nami, Rie Koizumi

Research output: Contribution to journalArticle

6 Citations (Scopus)

Abstract

The importance of sample size, although widely discussed in the literature on structural equation modeling (SEM), has not been widely recognized among applied SEM researchers. To narrow this gap, we focus on second language testing and learning studies and examine the following: (a) Is the sample size sufficient in terms of precision and power of parameters in a model using Monte Carlo analysis? (b) How are the results from Monte Carlo sample size analysis comparable with those from the N ≥ 100 rule and from the N: q ≥ 10 (sample size-free parameter ratio) rule? Regarding (a), parameter bias, standard error bias, coverage, and power were overall satisfactory, suggesting that sample size for SEM models in second language testing and learning studies is generally appropriate. Regarding (b), both rules were often inconsistent with the Monte Carlo analysis, suggesting that they do not serve as guidelines for sample size. We encourage applied SEM researchers to perform Monte Carlo analyses to estimate the requisite sample size of a model.

Original languageEnglish
Pages (from-to)329-353
Number of pages25
JournalInternational Journal of Testing
Volume13
Issue number4
DOIs
Publication statusPublished - 2013 Oct

Fingerprint

structural model
Testing
language
learning
trend
coverage

Keywords

  • Monte Carlo
  • power
  • precision
  • sample size
  • second language studies
  • structural equation modeling

ASJC Scopus subject areas

  • Education
  • Modelling and Simulation
  • Social Psychology

Cite this

Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research : A Monte Carlo Approach. / In'nami, Yo; Koizumi, Rie.

In: International Journal of Testing, Vol. 13, No. 4, 10.2013, p. 329-353.

Research output: Contribution to journalArticle

@article{b170ff6ff02c4c369fa1b23b7746e167,
title = "Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research: A Monte Carlo Approach",
abstract = "The importance of sample size, although widely discussed in the literature on structural equation modeling (SEM), has not been widely recognized among applied SEM researchers. To narrow this gap, we focus on second language testing and learning studies and examine the following: (a) Is the sample size sufficient in terms of precision and power of parameters in a model using Monte Carlo analysis? (b) How are the results from Monte Carlo sample size analysis comparable with those from the N ≥ 100 rule and from the N: q ≥ 10 (sample size-free parameter ratio) rule? Regarding (a), parameter bias, standard error bias, coverage, and power were overall satisfactory, suggesting that sample size for SEM models in second language testing and learning studies is generally appropriate. Regarding (b), both rules were often inconsistent with the Monte Carlo analysis, suggesting that they do not serve as guidelines for sample size. We encourage applied SEM researchers to perform Monte Carlo analyses to estimate the requisite sample size of a model.",
keywords = "Monte Carlo, power, precision, sample size, second language studies, structural equation modeling",
author = "Yo In'nami and Rie Koizumi",
year = "2013",
month = "10",
doi = "10.1080/15305058.2013.806925",
language = "English",
volume = "13",
pages = "329--353",
journal = "International Journal of Testing",
issn = "1530-5058",
publisher = "Routledge",
number = "4",

}

TY - JOUR

T1 - Review of Sample Size for Structural Equation Models in Second Language Testing and Learning Research

T2 - A Monte Carlo Approach

AU - In'nami, Yo

AU - Koizumi, Rie

PY - 2013/10

Y1 - 2013/10

N2 - The importance of sample size, although widely discussed in the literature on structural equation modeling (SEM), has not been widely recognized among applied SEM researchers. To narrow this gap, we focus on second language testing and learning studies and examine the following: (a) Is the sample size sufficient in terms of precision and power of parameters in a model using Monte Carlo analysis? (b) How are the results from Monte Carlo sample size analysis comparable with those from the N ≥ 100 rule and from the N: q ≥ 10 (sample size-free parameter ratio) rule? Regarding (a), parameter bias, standard error bias, coverage, and power were overall satisfactory, suggesting that sample size for SEM models in second language testing and learning studies is generally appropriate. Regarding (b), both rules were often inconsistent with the Monte Carlo analysis, suggesting that they do not serve as guidelines for sample size. We encourage applied SEM researchers to perform Monte Carlo analyses to estimate the requisite sample size of a model.

AB - The importance of sample size, although widely discussed in the literature on structural equation modeling (SEM), has not been widely recognized among applied SEM researchers. To narrow this gap, we focus on second language testing and learning studies and examine the following: (a) Is the sample size sufficient in terms of precision and power of parameters in a model using Monte Carlo analysis? (b) How are the results from Monte Carlo sample size analysis comparable with those from the N ≥ 100 rule and from the N: q ≥ 10 (sample size-free parameter ratio) rule? Regarding (a), parameter bias, standard error bias, coverage, and power were overall satisfactory, suggesting that sample size for SEM models in second language testing and learning studies is generally appropriate. Regarding (b), both rules were often inconsistent with the Monte Carlo analysis, suggesting that they do not serve as guidelines for sample size. We encourage applied SEM researchers to perform Monte Carlo analyses to estimate the requisite sample size of a model.

KW - Monte Carlo

KW - power

KW - precision

KW - sample size

KW - second language studies

KW - structural equation modeling

UR - http://www.scopus.com/inward/record.url?scp=84884302607&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84884302607&partnerID=8YFLogxK

U2 - 10.1080/15305058.2013.806925

DO - 10.1080/15305058.2013.806925

M3 - Article

AN - SCOPUS:84884302607

VL - 13

SP - 329

EP - 353

JO - International Journal of Testing

JF - International Journal of Testing

SN - 1530-5058

IS - 4

ER -