Can structural equation models in second language testing and learning research be successfully replicated?

Yo In'nami, Rie Koizumi

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Because structural equation models are widely used in testing and assessment, investigation into the accuracy of such models may help raise awareness of the value of reanalysis or replication. We focused on second language testing and learning studies and examined: (a) To what extent is information necessary for replication provided by authors? (b) To what extent can the original models be successfully replicated? Regarding (a), we e-mailed authors of 31 articles that did not contain information needed to replicate the study and asked them for the missing information. We obtained data from only four authors. Regarding (b), we succeeded in replicating 89% of the models in the preliminary analysis, 87% to 100% of fit indices, and 94% of parameter estimates. The results suggest that for the most part, structural equation modeling research reported in second language testing and learning research is accurate.

Original languageEnglish
Pages (from-to)262-273
Number of pages12
JournalInternational Journal of Testing
Volume10
Issue number3
DOIs
Publication statusPublished - 2010 Jul

Fingerprint

structural model
Testing
language
learning
Values

Keywords

  • Replication
  • Second language learning
  • Second language testing
  • Structural equation modeling

ASJC Scopus subject areas

  • Education
  • Modelling and Simulation
  • Social Psychology

Cite this

Can structural equation models in second language testing and learning research be successfully replicated? / In'nami, Yo; Koizumi, Rie.

In: International Journal of Testing, Vol. 10, No. 3, 07.2010, p. 262-273.

Research output: Contribution to journalArticle

@article{4b7163dc6ecb42f29cce3d81a7f8aeee,
title = "Can structural equation models in second language testing and learning research be successfully replicated?",
abstract = "Because structural equation models are widely used in testing and assessment, investigation into the accuracy of such models may help raise awareness of the value of reanalysis or replication. We focused on second language testing and learning studies and examined: (a) To what extent is information necessary for replication provided by authors? (b) To what extent can the original models be successfully replicated? Regarding (a), we e-mailed authors of 31 articles that did not contain information needed to replicate the study and asked them for the missing information. We obtained data from only four authors. Regarding (b), we succeeded in replicating 89{\%} of the models in the preliminary analysis, 87{\%} to 100{\%} of fit indices, and 94{\%} of parameter estimates. The results suggest that for the most part, structural equation modeling research reported in second language testing and learning research is accurate.",
keywords = "Replication, Second language learning, Second language testing, Structural equation modeling",
author = "Yo In'nami and Rie Koizumi",
year = "2010",
month = "7",
doi = "10.1080/15305058.2010.482219",
language = "English",
volume = "10",
pages = "262--273",
journal = "International Journal of Testing",
issn = "1530-5058",
publisher = "Routledge",
number = "3",

}

TY - JOUR

T1 - Can structural equation models in second language testing and learning research be successfully replicated?

AU - In'nami, Yo

AU - Koizumi, Rie

PY - 2010/7

Y1 - 2010/7

N2 - Because structural equation models are widely used in testing and assessment, investigation into the accuracy of such models may help raise awareness of the value of reanalysis or replication. We focused on second language testing and learning studies and examined: (a) To what extent is information necessary for replication provided by authors? (b) To what extent can the original models be successfully replicated? Regarding (a), we e-mailed authors of 31 articles that did not contain information needed to replicate the study and asked them for the missing information. We obtained data from only four authors. Regarding (b), we succeeded in replicating 89% of the models in the preliminary analysis, 87% to 100% of fit indices, and 94% of parameter estimates. The results suggest that for the most part, structural equation modeling research reported in second language testing and learning research is accurate.

AB - Because structural equation models are widely used in testing and assessment, investigation into the accuracy of such models may help raise awareness of the value of reanalysis or replication. We focused on second language testing and learning studies and examined: (a) To what extent is information necessary for replication provided by authors? (b) To what extent can the original models be successfully replicated? Regarding (a), we e-mailed authors of 31 articles that did not contain information needed to replicate the study and asked them for the missing information. We obtained data from only four authors. Regarding (b), we succeeded in replicating 89% of the models in the preliminary analysis, 87% to 100% of fit indices, and 94% of parameter estimates. The results suggest that for the most part, structural equation modeling research reported in second language testing and learning research is accurate.

KW - Replication

KW - Second language learning

KW - Second language testing

KW - Structural equation modeling

UR - http://www.scopus.com/inward/record.url?scp=79960485169&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79960485169&partnerID=8YFLogxK

U2 - 10.1080/15305058.2010.482219

DO - 10.1080/15305058.2010.482219

M3 - Article

AN - SCOPUS:79960485169

VL - 10

SP - 262

EP - 273

JO - International Journal of Testing

JF - International Journal of Testing

SN - 1530-5058

IS - 3

ER -