Transfer Learning and Distance Metric Learning in Non-stationary Environments




Journal Title

Journal ISSN

Volume Title



The development of various online platforms such as social media provide users platforms to share their thoughts, feedback and their daily lives. With the widespread of microblogs such as Twitter and the tremendous increase of the online social media data, the supervised models may suffer due to the label scarcity issue and the large variety of data domains. The cost of gaining true labels leads us to a practically important problem: can the labeled data from other related sources help predict the target task? In this thesis, the methods we propose to solve this task can be broadly divided into two ways: transfer learning based and metric learning based methods. For the first way, we perform online data mining in a scenario involving two types of independent data streams continuously generated from different domains. The non-stationary stream generated with labeled data instances is referred to as the Source Stream. Another independent data stream is assumed to be generated without label information, which is referred to as the Target Stream. We propose two efficient solutions for the multistream data mining framework by integrating change detection into online data stream adaptation. Our goal is to predict the truth value of data instances in the target stream by using the sufficient amount of label information in the different but related source stream. Also, due to the theoretical infinite length of data stream, the concept of both source and target stream may change over time, resulting in asynchronous data distribution drift. In this thesis, the concept drifts are continuously being addressed at the same time as we address covariate shift between the two independent streams. For the second way, after noticing the non-linearity and high-dimensionality of real-world data, we leverage a simple yet efficient solution, metric learning, to model the similarity relationships between data instances. The intuition behind is similar to the clustering methods: a given data point should share the same labels as its nearest neighbors, whereas data points of different labels should be far from the given point. Here, we propose a novel personalized hashtag retrieval framework leveraging the concept of metric-based meta learning. We perform this task in a label scarcity setting involving two distinct sets of users, meta-train set and meta-test set. Our goal is to learn how to recommend user-specific hashtags from meta-train set and adapt the knowledge to meta-test set with small sample sizes. We also design an adaptive multi-region framework to classify the abnormal part (lesion) in CT scan images using metric learning. This multi-region framework allows us to not only learn the local lesion similarity but also learn the overall image similarity adaptively.



Machine learning, Transfer learning (Machine learning), Data mining