Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing

Paolo Pedinotti; Emmanuele Chersoni; Enrico Santus; Alessandro Lenci

Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing

Paolo Pedinotti, Emmanuele Chersoni, Enrico Santus, Alessandro Lenci

Department of Chinese and Bilingual Studies

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

Abstract

An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans’ behavior.

Original language	English
Title of host publication	Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language
Editors	Valentina Pyatkin, Daniel Fried, Talita Anthonio
Publisher	Association for Computational Linguistics (ACL)
Pages	8–16
Number of pages	9
ISBN (Print)	978-1-955917-92-6
Publication status	Published - Jul 2022
Event	The Second Workshop on Understanding Implicit and Underspecified Language - Hyatt Regency Seattle, Seattle, United States Duration: 15 Jul 2022 → 15 Jul 2022 https://unimplicit2022.github.io/

Conference

Conference	The Second Workshop on Understanding Implicit and Underspecified Language
Abbreviated title	Unimplicit 2022
Country/Territory	United States
City	Seattle
Period	15/07/22 → 15/07/22
Internet address	https://unimplicit2022.github.io/

More information

https://aclanthology.org/2022.unimplicit-1.2.pdf

Cite this

Pedinotti, P., Chersoni, E., Santus, E., & Lenci, A. (2022). Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing. In V. Pyatkin, D. Fried, & T. Anthonio (Eds.), Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language (pp. 8–16). Association for Computational Linguistics (ACL). https://aclanthology.org/2022.unimplicit-1.2.pdf

@inproceedings{61e848ab3c584de395a5c17c55e2c4f9,

title = "Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing",

abstract = "An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans{\textquoteright} behavior.",

author = "Paolo Pedinotti and Emmanuele Chersoni and Enrico Santus and Alessandro Lenci",

year = "2022",

month = jul,

language = "English",

isbn = "978-1-955917-92-6",

pages = "8–16",

editor = "Valentina Pyatkin and Daniel Fried and Talita Anthonio",

booktitle = "Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language",

publisher = "Association for Computational Linguistics (ACL)",

note = "The Second Workshop on Understanding Implicit and Underspecified Language, Unimplicit 2022 ; Conference date: 15-07-2022 Through 15-07-2022",

url = "https://unimplicit2022.github.io/",

}

Pedinotti, P, Chersoni, E, Santus, E & Lenci, A 2022, Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing. in V Pyatkin, D Fried & T Anthonio (eds), Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language. Association for Computational Linguistics (ACL), pp. 8–16, The Second Workshop on Understanding Implicit and Underspecified Language, Seattle, United States, 15/07/22. <https://aclanthology.org/2022.unimplicit-1.2.pdf>

Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing. / Pedinotti, Paolo; Chersoni, Emmanuele; Santus, Enrico et al.
Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language. ed. / Valentina Pyatkin; Daniel Fried; Talita Anthonio. Association for Computational Linguistics (ACL), 2022. p. 8–16.

Research output: Chapter in book / Conference proceeding › Conference article published in proceeding or book › Academic research › peer-review

TY - GEN

T1 - Pragmatic and Logical Inferences in NLI Systems

T2 - The Second Workshop on Understanding Implicit and Underspecified Language

AU - Pedinotti, Paolo

AU - Chersoni, Emmanuele

AU - Santus, Enrico

AU - Lenci, Alessandro

PY - 2022/7

Y1 - 2022/7

N2 - An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans’ behavior.

AB - An intelligent system is expected to perform reasonable inferences, accounting for both the literal meaning of a word and the meanings a word can acquire in different contexts. A specific kind of inference concerns the connective and, which in some cases gives rise to a temporal succession or causal interpretation in contrast with the logic, commutative one (Levinson, 2000). In this work, we investigate the phenomenon by creating a new dataset for evaluating the interpretation of and by NLI systems, which we use to test three Transformer-based models. Our results show that all systems generalize patterns that are consistent with both the logical and the pragmatic interpretation, perform inferences that are inconsistent with each other, and show clear divergences with both theoretical accounts and humans’ behavior.

M3 - Conference article published in proceeding or book

SN - 978-1-955917-92-6

SP - 8

EP - 16

BT - Proceedings of The Second Workshop on Understanding Implicit and Underspecified Language

A2 - Pyatkin, Valentina

A2 - Fried, Daniel

A2 - Anthonio, Talita

PB - Association for Computational Linguistics (ACL)

Y2 - 15 July 2022 through 15 July 2022

ER -

Pragmatic and Logical Inferences in NLI Systems: The Case of Conjuction Buttressing

Abstract

Conference

More information

Fingerprint

Cite this