Abstract
This presentation reports the findings from a trial of the AI Review Assessment System (AIReAS), an automated writing evaluation platform powered by GPT-4 and developed by the Hong Kong Polytechnic University. AIReAS enables teachers to input assignment details and assessment rubrics, generating a set of rubric-based feedback and a predicted grade for each student submission. The trial was conducted on a research report assignment in a general education course on the history of Hong Kong. Students received feedback from both teachers and the platform and were interviewed for comments on both types of feedback.
Thematic analysis of the focus group interview transcripts showed that students appreciated the AI feedback for its wide coverage, as well as its high efficiency and accessibility. However, they found the feedback too general to be helpful for improving their reports. In contrast, teacher feedback was perceived as more detailed and useful as it included specific examples of issues. Students showed a clear preference for teacher feedback but expressed interest in using both types to maximize their benefits. The results confirmed the value of AI feedback and its potential limitations as identified in research on automated writing evaluation.
Thematic analysis of the focus group interview transcripts showed that students appreciated the AI feedback for its wide coverage, as well as its high efficiency and accessibility. However, they found the feedback too general to be helpful for improving their reports. In contrast, teacher feedback was perceived as more detailed and useful as it included specific examples of issues. Students showed a clear preference for teacher feedback but expressed interest in using both types to maximize their benefits. The results confirmed the value of AI feedback and its potential limitations as identified in research on automated writing evaluation.
| Original language | English |
|---|---|
| Publication status | Published - 2024 |
| Event | 4th International Conference on English Across the Curriculum - The Hong Kong Polytechnic University , Hong Kong, Hong Kong Duration: 2 Dec 2024 → 4 Dec 2024 https://www.polyu.edu.hk/en/eacconference/ |
Conference
| Conference | 4th International Conference on English Across the Curriculum |
|---|---|
| Country/Territory | Hong Kong |
| City | Hong Kong |
| Period | 2/12/24 → 4/12/24 |
| Internet address |