Efficient and Effective Structure Learning of Graphical Models

To achieve the full potential of AI system, it is necessary that they are able to learn in structured, noisy domains in the presence of humans. Extensive research has been done for faithful modeling and learning in complex domains for predictive analytics. However, there are several unresolved challenges w.r.t scalability, interpretability, and the ability of human to influence the speed and effectiveness of learning. In this dissertation, we aim to address and relax the key assumptions that most methods make, including (a) flat feature vectors for data representation, (b) scale of the data, (c) high quality of examples, and (d) humans’ involvement as mere labelers. The objective of this dissertation is to develop algorithms that (1) model the domain structure faithfully, (2) handle uncertainty explicitly, and (3) facilitate domain expert interaction seamlessly. To this effect, we develop efficient learning methods using probabilistic models as the underlying representation, which are interpretable and can allow rich human-in-the-loop interactions. More specifically, this dissertation aims to tackle two complementary challenges that are to be addressed as an important step in realizing the full potential of AI. They are 1) predictive modeling from observational data and 2) predictive modeling with rich human inputs. For predictive modeling from observational data, we consider learning from noisy and potentially structured data. This class of algorithms considers problems that arise in dense data sets with noise and uncertainty. Learning effectively in complex structured domains with high dimensional feature space and rich relations can be challenging. To address these problems, we adapt the successful machinery of functional gradient boosting in learning probabilistic (logic) models that scale to large volumes of data with improved performance making it attractive for large-scale inference tasks. We successfully applied this class of methods to effectively learn in different settings: (i) discriminative Bayesian Networks, (ii) directed models in structured domains and, (iii) tractable probabilistic circuits. We combine the benefits of probabilistic models with machine learning methods and propose approaches to learn Causal Bayesian Networks that are scalable and effective in unearthing the ground graphs. In knowledge-rich and data-poor domains such as medicine, it is becoming increasingly important for the domain experts to steer the learning algorithms. Predictive modeling in the presence of a human expert address this problem. To this effect, we build algorithms that can (a) take rich domain knowledge in addition to the examples, (b) enable richer interaction by developing innovative interfaces for agent-human communication, and (c) query an expert to improve the model for a particular predictive task at hand. We investigate our human-in-the-loop approaches to learning effectively in the following settings: (i) guide the transfer learning algorithm across seemingly unrelated domains and (ii) active feature subset elicitation in structured domains. In summary, we believe that this dissertation takes a significant step towards building a robust AI agent by addressing challenges that exist when learning from noisy, structured data in the presence of a human expert.