Automated Essay Scoring: Argument Persuasiveness
Despite being investigated for over 50 years, the task of Automated Essay Scoring (AES) is far from being solved. Nevertheless, it continues to draw a lot of attention in the natural language processing community in part because of its commercial and educational values as well as the research challenges it brings about. While the majority of work on AES has focused on evaluating an essay’s overall quality, comparatively less work has focused on scoring an essay along specific dimensions of quality. Among many dimensions, argument persuasiveness is arguably the most important but largely ignored dimension. In this thesis, we focus on argument persuasiveness scoring. First, we present our publicly available corpus for argument persuasiveness. Our corpus is the first corpus of essays that are simultaneously annotated with argument components, argument persuasiveness scores, and attributes of argument components that impact an argument’s persuasiveness. The inter-annotator agreement and the oracle experiments indicate that our annotations are reliable and the attributes can help predict the persuasiveness score. Second, we present the first set of neural models that predict the persuasiveness of an argument and its attributes in a student essay, enabling useful feedback to be provided to students on why their arguments are (un)persuasive in addition to how persuasive they are. The evaluation of our models shows that automatically computed attributes are useful for persuasiveness scoring and that performance can be improved by improving attribute prediction.