Parametric analyses of summative scores may lead to conflicting inferences when comparing groups: A simulation study

Asaduzzaman Khan, Chi-Wen Chien, Karl S. Bagraith

Research output: Journal article publicationJournal articleAcademic researchpeer-review

4 Citations (Scopus)

Abstract

Objective: To investigate whether using a parametric statistic in comparing groups leads to different conclusions when using summative scores from rating scales compared with using their corresponding Rasch-based measures. Methods: A Monte Carlo simulation study was designed to examine between-group differences in the change scores derived from summative scores from rating scales, and those derived from their corresponding Rasch-based measures, using 1-way analysis of variance. The degree of inconsistency between the 2 scoring approaches (i.e. summative and Rasch-based) was examined, using varying sample sizes, scale difficulties and person ability conditions. Results: This simulation study revealed scaling artefacts that could arise from using summative scores rather than Rasch-based measures for determining the changes between groups. The group differences in the change scores were statistically significant for summative scores under all test conditions and sample size scenarios. However, none of the group differences in the change scores were significant when using the corresponding Rasch-based measures. Conclusion: This study raises questions about the validity of the inference on group differences of summative score changes in parametric analyses. Moreover, it provides a rationale for the use of Rasch-based measures, which can allow valid parametric analyses of rating scale data.
Original languageEnglish
Pages (from-to)300-304
Number of pages5
JournalJournal of Rehabilitation Medicine
Volume47
Issue number4
DOIs
Publication statusPublished - 1 Jan 2015

Keywords

  • Parametric statistics
  • Rasch analysis
  • Rasch-based measures
  • Rating scales
  • Simulation
  • Summative scores

ASJC Scopus subject areas

  • Physical Therapy, Sports Therapy and Rehabilitation
  • Rehabilitation

Cite this