Examining the inter-rater and test–retest reliability of the student practice evaluation form-revised (SPEF-R) for occupational therapy students

Sylvia Rodger, Allison Coleman, Anne Maree Caine, Chi-Wen Chien, Jodie Copley, Merrill Turpin, Ted Brown

Research output: Journal article publicationJournal articleAcademic researchpeer-review

9 Citations (Scopus)

Abstract

Background/aim: The Student Practice Evaluation Form-Revised Edition (SPEF-R) is widely used in Australian universities to evaluate occupational therapy students’ performance in practice education. Reliable completion of the SPEF-R by practice educators is critical for students and universities from a quality assurance perspective. This study used standardised video vignettes to examine the inter-rater and test–retest reliability of practice educators when completing the SPEF-R. Methods: Nine vignettes were developed with three levels of student performance (below expectations, adequate-toproficient, and outstanding) for each of three scenarios which depicted a multidisciplinary team meeting, a supervision session, and a home visit respectively. Seventy-five occupational therapy practice educators viewed the vignettes and completed an online survey to rate student performance in each vignette using five selected SPEF-R items and using the five-point rating scale. Twenty of these practice educators completed the rating process twice for test–retest reliability purposes. Percentage agreement, t-tests and Rasch Measurement Model were used for analysis. Results: Practice educators exhibited consistent ratings on most of the SPEF-R items, except for rating adequateto-proficient student performance on three items for interrater reliability and eight items for test–retest reliability. Rasch analysis found that the majority of practice educators (96–98.7%) exhibited consistency in their use of the SPEF-R rating scales. Practice educators also demonstrated satisfactory test–retest agreement of severity/leniency in rating student performance in the multidisciplinary team meeting scenario. Conclusion: The study findings suggest that the SPEF-R could be used reliably and interpreted consistently by practice educators with diverse backgrounds and levels of experience.
Original languageEnglish
Pages (from-to)353-363
Number of pages11
JournalAustralian Occupational Therapy Journal
Volume61
Issue number5
DOIs
Publication statusPublished - 1 Oct 2014
Externally publishedYes

Keywords

  • Education
  • Occupational therapy
  • Professional
  • Reproducibility of results

ASJC Scopus subject areas

  • Occupational Therapy

Cite this