Chasing the Unicorn? The Feasibility of Automatic Assessment of Interpreting Fluency

Research output: Chapter in book / Conference proceedingChapter in an edited book (as author)Academic researchpeer-review

Abstract

This chapter examines the feasibility of using Praat for automatic assessment of interpreting fluency (i.e., speed and breakdown fluency). A total of 140 audio recordings were collected in an English-to-Chinese consecutive interpreting exam. Two raters assessed the fluency of the interpreting performance using a four-point scale and their mean scores were used as the perceived fluency scores. Two versions of the audio files, unedited and edited (with inter-segmental silences and noises being removed), were subjected to Praat analysis in two conditions (intensity-adjusted versus intensity-unadjusted). In the intensity-adjusted condition, the intensity threshold was set differently for individual recordings, depending on the maximum intensity and the 99% quantile intensity. In the intensity-unadjusted condition, the threshold was set uniformly for all the audio files. The correlations between Speed Fluency, Breakdown Fluency, and Perceived Fluency were examined across the four conditions (i.e., edited and intensity-adjusted, edited and intensity-unadjusted, unedited and intensity-adjusted, unedited and intensity-unadjusted). Statistical analyses showed that: (a) removing inter-segmental silences and noises slightly improved the correlation coefficients; (b) adjusting the intensity threshold significantly improved the correlation coefficients; (c) the silent pause threshold of 0.25 second produced the best correlation coefficients; and (d) Mean Silence Duration had the strongest correlation with judged fluency. Based on these findings, a flowchart is designed to guide teachers’ decision-making in fluency assessment, ranging from pre-processing of audio files, to configuration of parameters in Praat, and finally to relating acoustic measures to assessment goals.

Original languageEnglish
Title of host publicationTesting and Assessment of Interpreting
PublisherSpringer Science and Business Media Deutschland GmbH
Pages143-158
Number of pages16
DOIs
Publication statusPublished - Apr 2021

Publication series

NameNew Frontiers in Translation Studies
ISSN (Print)2197-8689
ISSN (Electronic)2197-8697

Keywords

  • Automatic assessment
  • Breakdown fluency
  • Interpreting performance
  • Praat
  • Speed fluency

ASJC Scopus subject areas

  • Linguistics and Language
  • Education
  • Language and Linguistics
  • Communication

Cite this