Commonsense Reasoning With Discourse Relations

Date

2022-05-11

Journal Title

Journal ISSN

Volume Title

Publisher

item.page.doi

Abstract

To understand language, machines need to be cognizant of the implied commonsense knowledge underlying the text. Observations obtained from real-world scenarios contain richer and more detailed information than what natural language typically conveys. In such cases, it is imperative for language understanding systems to make accurate inferences about commonsense knowledge to fill in what is being assumed. While performing commonsense reasoning over text is trivial for most humans, it remains an elusive goal for NLP and more broadly AI systems. Moreover, systems that struggle to perform commonsense reasoning in the desired manner risk producing redundant and even harmful outputs. To address the underlying challenges, it is necessary to understand how the task of commonsense reasoning has been dealt with pre-trained language models (PLMs), as well as the pitfalls and the strengths of PLMs to discern what kind of reasoning they are capable of performing. The closing gap between PLMs and human baselines on existing benchmarks suggests that there is a requirement for a new one that can reliably address the well-known limitations of PLMs. Given the importance and lack of work in discourse phenomena with PLMs and how current training objectives prove to be insufficient, we present a discourse relation-based benchmark that requires PLMs to reason over discourse markers and context to make the task intricate from a linguistic and commonsense standpoint. We hope that this benchmark would allow for newer commonsense reasoning methods to be developed in addition to serving as a resource that proves to be difficult for state-of-the-art PLMs. The first part of the thesis provides a comprehensive survey discussing the strengths and weaknesses of state-of-the-art pre-trained models for commonsense reasoning and generation. In the second part, we present DISCOSENSE, a novel benchmark for performing commonsense reasoning through understanding discourse relations. This benchmark addresses two major shortcomings with Pre-trained Language Models (PLMs), degradation in the reasoning capability of PLMs as the complexity of the task increases and difficulty in determining the most plausible ending when multiple scenarios are possible with a provided situation. We generate compelling distractors in DISCOSENSE using Conditional Adversarial Filtering, an extension of Adversarial Filtering that employs conditional text generation. We show that state-of-the-art models struggle to perform well on this task, making this an ideal benchmark for next-generation commonsense reasoning systems. We believe that this benchmark would contribute toward improving next-generation NLP systems for commonsense reasoning tasks. These sections are then followed by a discussion of avenues for future research.

Description

Keywords

Commonsense reasoning, Modeling languages (Computer science), Computational linguistics, Artificial intelligence

item.page.sponsorship

Rights

Citation