Abstract
Prior studies on event knowledge in sentence comprehension have shown that the aspect of the main verb plays an important role in the processing of non-core semantic roles, such as locations: when the aspect of the main verb is
imperfective, locations become more salient in the mental representation of the event and are easier for human comprehenders to process.
In our study, we tested the popular language model BERT on two datasets derived from experimental studies to determine whether BERT’s predictions of prototypical event locations were also influenced by aspect. We found that, although BERT efficiently modelled the typicality of locations, it did so independently of the verb aspect. Even when the transformer was forced to focus on the verb phrase by masking the context words in the sentence, the typicality predictions were still accurate; in addition, we found aspect to have a stronger influence on the scores, with locations in the imperfective setting being associated with lower surprisal values.
imperfective, locations become more salient in the mental representation of the event and are easier for human comprehenders to process.
In our study, we tested the popular language model BERT on two datasets derived from experimental studies to determine whether BERT’s predictions of prototypical event locations were also influenced by aspect. We found that, although BERT efficiently modelled the typicality of locations, it did so independently of the verb aspect. Even when the transformer was forced to focus on the verb phrase by masking the context words in the sentence, the typicality predictions were still accurate; in addition, we found aspect to have a stronger influence on the scores, with locations in the imperfective setting being associated with lower surprisal values.
| Original language | English |
|---|---|
| Title of host publication | Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021 |
| Editors | Chengqing Zong, Fei Xia, Wenjie Li, Roberto Navigli |
| Publisher | Association for Computational Linguistics (ACL) |
| Pages | 2922-2929 |
| Number of pages | 8 |
| Publication status | Published - Aug 2021 |
| Event | The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021) - Online Duration: 1 Aug 2021 → 6 Aug 2021 https://2021.aclweb.org/ |
Conference
| Conference | The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021) |
|---|---|
| Abbreviated title | ACL-IJCNLP 2021 |
| Period | 1/08/21 → 6/08/21 |
| Internet address |