Learning Tractable Probabilistic Graphical Models in Continuous and Temporal Domains

Probabilistic Graphical Models (PGMs) such as Bayesian networks (BNs) and Markov random fields (MRFs) present a powerful framework for representing complex probabilistic dependencies and reasoning about uncertainties. However, inference over these models is typically intractable. Recently, tractable models such as cutset networks and sum-product networks (SPNs) have become increasingly popular because they allow some inferences to be performed in polynomial time with respect to the size of the network. However, most of this recent work focuses on discrete domains, and the resulting models can yield poor predictive performance in practice as a result of restrictive modeling assumptions. In temporal modeling problems, existing probabilistic models such as dynamic Bayesian networks (DBNs), hidden Markov networks (HMMs), dynamic sum product networks (DSPNs), and dynamic cutset networks (DCNs) employ first-order Markov and stationary assumptions, and limit representational power so that efficient (approximate) inference procedures can be applied. In this work, we aim to design a general probabilistic framework for continuous, temporal modeling that allows efficient and accurate approximate inference. To this end, we first extend the idea of cutset conditioning into continuous domains and propose a probabilistic model that encodes the joint distribution as the product of a local, complex distribution over a small subset of variables and a fully tractable conditional distribution whose pa rameters are controlled by a neural network. This model admits exact inference when all variables in the local distribution are observed, otherwise we show that “cutset” sampling can be employed to efficiently generate accurate predictions in practice. We then extend our framework into temporal domains and model the full transition distribution as a tractable continuous density over the variables at the current time slice only, while the parameters are controlled using a Recurrent Neural Network (RNN) that takes all previous observations as input. We show that, in this model, various inference tasks can be efficiently implemented using forward filtering with simple gradient ascent. Lastly, we enhance the robustness of our continuous tractable model against distribution shifts using the Distributionally Robust Supervised Learning (DRSL) framework. We demonstrate the approach of applying DRSL in learning robust generative probabilistic models and develop an efficient linearithmic algorithm for addressing the adversarial risk minimization problem. We evaluated our models’ predictive performance and robustness through various tasks on real-world datasets, and the experimental results demonstrate the superior performance of our models against existing competitors.