|dc.description.abstract||Text understanding is a key component of Natural Language Processing (NLP) and Artificial Intelligence (AI). The traditional approaches to solve this problem are by joining the
features selected from sentences and the relevant world knowledge together to make further
inference. However, such feature based methods not only require a great deal of human
effort, but also can only represent the lexical and syntactic information of text and the semantic meanings of text cannot be well represented. Hence, we present novel neural language
models that represent text as semantic vector and apply them to solve multiple high-level
text understanding tasks. The neural language models not only provide effective and general
representations for learning the semantics of text, but also can obtain state-of-the-art performance on a variety of text understanding tasks. In this dissertation, we studied several
Deep Learning techniques, including multi-task learning, transfer learning and multi-lingual
learning and applied them to solve a variety of text understanding tasks, such as Semantic
Textual Similarity, Textual Entailment and Semantic Relation Extraction.
First, we explored the multi-task learning models for Semantic Relatedness task and Textual Entailment task. In Natural Language Processing, there exist several tasks that are
highly related to each other. Instead of using single task learning that optimize each task
specific system independently, we can optimize all tasks simultaneously using one multi-task
learning model. We selected two related NLP tasks, Semantic Relatedness task and Textual
Entailment task, and trained them jointly with a variety of multi-task learning models. This
study explored if multi-task learning can outperform the performance of single-task learning
on related NLP tasks.
Second, we studied the transfer learning models for Semantic Relation Classification task.
Domain adaptation is a common issue in Natural Language Processing, due to the fact that
creating corpora in novelty domain requires a large human annotation effort. In neural
language models, researchers made several attempts to find universal sentence embedding
methods, aiming to obtain general-purpose sentence embeddings that can be widely adopted
to a wide range of NLP tasks. We conducted an experiment to evaluate if transfer learning
can help to train general-purpose sentence encoders for Relation Classification task using a
limited-size of training data.
Last, we studied the neural language models in multi-lingual domains. Specifically, we
created models to solve the Semantic Relation Extraction task for Chinese and Semantic
Textual Similarity task for Arabic, English and Spanish. We aim to show that, compared
to traditional feature based methods, neural language models can (1) achieve state-of-theart performance without any or very few manually designed features, and (2) learn general
sentence representation regardless of languages and domains.||